ruMMLU-pro

# ruMMLU-Pro ## 📝 Dataset Summary **ruMMLU-Pro** is a high-quality Russian translation of the MMLU-Pro dataset, containing **11,986** examples. The dataset preserves the full structure, labeling scheme, answer indices, and chain-of-thought content from the original benchmark while providing localized Russian text suitable for evaluating multilingual and Russian-native language models. **ruMMLU-Pro** enables rigorous assessment of Russian-native and multilingual language models on high-level reasoning tasks, offering strong coverage of domains such as medicine, law, business, STEM disciplines, humanities, and more. --- ## 📁 Dataset Structure ### **Data Fields** | Field | Type | Description | |----------------|------------|-------------| | `question_id` | `int` | Unique question identifier | | `question` | `string` | Russian-translated question prompt | | `options` | `list[str]`| List of answer options | | `answer` | `string` | Correct answer label (typically A/B/C/D or similar) | | `answer_index` | `int` | Index of the correct option | | `cot_content` | `string` | Chain-of-thought explanation supporting the answer | | `category` | `string` | Domain category label | | `src` | `string` | Provenance reference to original English task | ### **Example Entry** ```json {"question_id": 262, "question": "Как называется скорость, с которой рынок принимает инновацию?", "options": ["Скорость проникновения на рынок.", "Процесс принятия.", "Шкала принятия рынком.", "Процесс насыщения рынка.", "Процесс дифференциации.", "Жизненный цикл продукта.", "Процесс диффузии.", "Скорость инноваций.", "Процесс диверсификации.", "Жизненный цикл принятия."], "answer": "G", "answer_index": 6, "cot_content": "", "category": "business", "src": "ori_mmlu-marketing" } ```