mygitphase/mmlu_indic

Indic MMLU数据集是多语言版本的Massive Multitask Language Understanding（MMLU）基准测试，从英语翻译成10种印度语言。该版本仅包含开发集和测试集的翻译，涵盖孟加拉语、古吉拉特语、印地语、卡纳达语、马拉地语、马拉雅拉姆语、奥里亚语、旁遮普语、泰米尔语和泰卢固语。每个示例是一个多项选择题，包含目标语言的问题文本、四个可能答案的列表（A、B、C、D）、正确答案索引（0-3）和语言代码。数据集用于评估模型在多语言环境下的语言理解能力，但存在技术术语翻译可能不精确、某些主题（如美国法律）缺乏直接对应概念以及文化和教育体系差异可能影响问题相关性等限制。

Indic MMLU is a multilingual version of the Massive Multitask Language Understanding (MMLU) benchmark, translated from English into 10 Indian languages. This version contains translations of the development and test sets only, covering Bengali, Gujarati, Hindi, Kannada, Marathi, Malayalam, Oriya, Punjabi, Tamil, and Telugu. Each example is a multiple-choice question containing question text in the target language, a list of four possible answers (A, B, C, D) in the target language, the correct answer index (0-3), and a language code. The dataset is used to evaluate model language understanding in multilingual contexts, with known limitations including potential imprecision in technical term translation, lack of direct equivalents for some subjects (e.g., US Law), and cultural and educational system differences that may affect question relevance.