awinml/medmcqa

---
language:
- en
license: apache-2.0
pretty_name: MedMCQA (AIIMS & NEET PG Medical Entrance MCQs)
size_categories:
- 100K<n<1M
task_categories:
- question-answering
language_creators:
- expert-generated
annotations_creators:
- expert-generated
multilinguality:
- monolingual
tags:
- medical
- neet-pg
- aiims
- multiple-choice
- benchmark
configs:
- config_name: default
default: true
data_files:
- split: train
path: train.json
- split: validation
path: dev.json
- split: test
path: test.json
---

# MedMCQA (AIIMS & NEET PG Medical Entrance MCQs)

## Dataset Summary

This dataset is a re-upload of the [MedMCQA](https://github.com/medmcqa/medmcqa) dataset introduced by Pal et al. in *[MedMCQA: A Large-Scale Multi-Subject Multi-Choice Dataset for Medical Domain Question Answering](https://arxiv.org/abs/2203.14371)* (ACL 2022).

MedMCQA is a large-scale multiple-choice question answering dataset sourced from Indian medical entrance examinations (AIIMS PG and NEET PG). It covers **20 medical subjects** and contains over **194,000** questions with four answer options each.

This repository includes:
- the **train** split (182,822 questions from mock tests and online test series),
- the **dev** split (4,183 questions from NEET PG exams, 2001–present),
- the **test** split (6,150 questions from AIIMS PG exams, 1991–present).

> **Note**: The test split has **no ground-truth labels**. The standard community practice is to evaluate on the **dev** split. To evaluate on the test set, predictions must be submitted via the [official submission form](https://forms.gle/xLJHNbuvaRa2FXbD8).

**Original resources**
| Resource | Link |
|---|---|
| Original repository | https://github.com/medmcqa/medmcqa |
| Published paper (ACL 2022) | https://aclanthology.org/2022.findings-acl.56 |
| arXiv preprint | https://arxiv.org/abs/2203.14371 |

## Supported Tasks

- **Multiple-choice question answering**: given a medical question and four answer options, predict the correct option.
- **Medical QA benchmarking**: evaluate domain-specific language models on multi-subject clinical and preclinical reasoning across 20 medical subjects.

## Languages

English (`en`)

## Dataset Structure

### Loading the Dataset

```python
from datasets import load_dataset

dataset = load_dataset("awinml/medmcqa")
```

### Data Splits

| Split | Examples | Has Labels (`cop`) | Has Explanations (`exp`) |
|---|---:|:---:|:---:|
| train | 182,822 | ✅ | ✅ (88% non-null) |
| validation | 4,183 | ✅ | ✅ (53% non-null) |
| test | 6,150 | ❌ | ❌ |
| **total** | **193,155** | | |

The splits are **separated by exam source**, not randomly:

| Split | Source Exam | Period |
|---|---|---|
| train | Mock tests & online test series | Various |
| validation (dev) | NEET PG exam | 2001–present |
| test | AIIMS PG exam | 1991–present |

### Data Fields

| Field | Type | Description |
|---|---|---|
| `id` | `string` | Unique question ID. |
| `question` | `string` | The question text. |
| `opa` | `string` | Option A text. |
| `opb` | `string` | Option B text. |
| `opc` | `string` | Option C text. |
| `opd` | `string` | Option D text. |
| `cop` | `int` | Correct option index (1–4). Present in train and validation only. |
| `exp` | `string` | Detailed medical explanation. Present in train and validation only (many null). |
| `subject_name` | `string` | Medical subject category (20 subjects). |
| `topic_name` | `string` | Topic within the subject. |
| `choice_type` | `string` | `"single"` or `"multi"` answer type. |

### Example

```json
{
"id": "f469cb22-2b04-4af1-8685-ad2831060a54",
"question": "Which of the following is not true about glelidings?",
"opa": "Gliding joints allow movement in a single plane",
"opb": "Gliding joints allow sliding movements",
"opc": "Gliding joints are also called plane joints",
"opd": "Gliding joints are found in the carpal bones of the wrist",
"cop": 1,
"exp": "Gliding joints allow sliding or gliding movements and are also called plane joints. They are found in the carpal bones of the wrist. They do not restrict movement to a single plane.",
"subject_name": "Anatomy",
"topic_name": "General anatomy",
"choice_type": "single"
}
```

### Subject Categories (20 total)

Anaesthesia, Anatomy, Biochemistry, Dental, ENT, Forensic Medicine, Gynaecology & Obstetrics, Medicine, Microbiology, Ophthalmology, Pathology, Pediatrics, Pharmacology, Physiology, Psychiatry, Radiology, Skin, Social & Preventive Medicine, Surgery, Unknown.

## Dataset Creation

### Source Data

This dataset is derived from the original [medmcqa/medmcqa](https://github.com/medmcqa/medmcqa) release. The questions are sourced from Indian medical entrance examinations (AIIMS PG and NEET PG), covering a broad range of medical subjects.

This re-upload preserves the original train/dev/test split and all fields from the original authors.

### Personal and Sensitive Information

The dataset does not contain real patient records or direct personal identifiers. Questions are written as medical knowledge MCQs and may reference clinical scenarios with demographic or health-related attributes.

## Considerations for Using the Data

### Out-of-Scope Use

This dataset should **not** be used for clinical diagnosis, treatment recommendations, or as a substitute for licensed medical expertise. Performance on multiple-choice exam questions does not reflect clinical safety.

### Limitations

- **India-centric**: sourced from Indian medical entrance exams (AIIMS PG, NEET PG), which may reflect curricula and terminology specific to the Indian medical education system.
- **Exam-style format**: multiple-choice exam performance does not necessarily reflect clinical usefulness.
- **No test labels**: the test split has no publicly available ground-truth labels; evaluation requires submission to the official form.
- **Incomplete explanations**: 12% of train and 47% of dev entries have null explanation fields.

## Licensing Information

The original [medmcqa/medmcqa](https://github.com/medmcqa/medmcqa) repository is distributed under the [Apache License 2.0](https://github.com/medmcqa/medmcqa/blob/main/LICENSE). This re-upload follows that license.

## Citation

If you use this dataset, please cite the original MedMCQA paper:

```bibtex
@inproceedings{pal2022medmcqa,
title = {MedMCQA: A Large-Scale Multi-Subject Multi-Choice Dataset for Medical Domain Question Answering},
author = {Pal, Ankit and Umapathi, Logesh Kumar and Sankarasubbu, Malaikannan},
booktitle = {Proceedings of the Conference on Health, Inference, and Learning (CHIL)},
series = {Proceedings of Machine Learning Research},
year = {2022},
url = {https://arxiv.org/abs/2203.14371}
}
```

## Dataset Curators

- **Original dataset authors**: Ankit Pal, Logesh Kumar Umapathi, Malaikannan Sankarasubbu
- **Hugging Face re-upload**: [awinml](https://huggingface.co/awinml)