MILU

MILU（Multi-task Indic Language Understanding Benchmark）是由AI4Bharat创建的综合性评估基准，旨在填补评估大型语言模型在印度语言能力方面的空白。该数据集涵盖8个领域和42个主题，跨越11种印度语言，包括科学、数学、艺术、法律等多个领域。数据集通过从印度各地区和州级考试中收集问题创建，确保了文化相关性和地域特色。MILU的应用领域广泛，旨在评估和提升模型在低资源语言和文化理解方面的能力，特别是在印度多样化的语言和文化背景下。MILU (Multi-task Indic Language Understanding Benchmark) is a comprehensive evaluation benchmark developed by AI4Bharat, aiming to fill the gap in evaluating the capabilities of Large Language Models (LLMs) in Indian languages. This benchmark covers 8 domains and 42 topics across 11 Indian languages, encompassing diverse fields such as science, mathematics, art, law and more. It is constructed by collecting questions from regional and state-level examinations across India, ensuring cultural relevance and regional distinctiveness. Boasting a wide range of application scenarios, MILU is designed to evaluate and enhance models’ abilities in low-resource language and cultural understanding, especially against the backdrop of India’s diverse linguistic and cultural landscape.

搜集汇总

背景与挑战

背景概述

MILU是一个多任务印度语言理解基准数据集，包含约80,000个多选题，覆盖11种印度语言和8个领域，特别强调印度文化和知识。数据集旨在评估大型语言模型在多样化和文化相关任务中的表现。