Indic MMLU

The Indic MMLU dataset is a multilingual adaptation of the Massive Multitask Language Understanding (MMLU) benchmark developed to evaluate the reasoning, knowledge comprehension, and multilingual capabilities of Large Language Models (LLMs) across Indian languages. The dataset consists of professionally translated and quality-filtered multiple-choice question-answer pairs spanning diverse academic and professional domains, including science, mathematics, history, law, medicine, engineering, humanities, and social sciences.
The primary purpose of this dataset is to provide a standardized benchmark for assessing model performance in low-resource and linguistically diverse Indic settings. The dataset enables research in multilingual NLP, cross-lingual transfer learning, language alignment, and culturally grounded AI evaluation.
The dataset was generated through a structured pipeline involving machine-assisted translation of the original English MMLU benchmark into selected Indic languages, followed by extensive quality filtering using translation evaluation metrics such as BLEU, chrF++, and TER. Additional validation steps were applied to preserve semantic fidelity, answer consistency, and linguistic fluency. The final data is provided in standardized machine-readable formats suitable for benchmarking and downstream evaluation workflows.
Indic MMLU is intended for researchers, academic institutions, and industry practitioners working on multilingual AI systems, Indic language technologies, and large-scale language model evaluation. By extending a widely recognized benchmark into Indian languages, the dataset contributes toward more inclusive, representative, and culturally relevant evaluation standards for modern AI systems