EtoUbivaetMnya1997/MedXpertQA-Exam

---
license: cc-by-4.0
task_categories:
- text-retrieval
language:
- en
tags:
- medical-retrieval
pretty_name: R2MED Benchmark
size_categories:
- 10M<n<100M
configs:
- config_name: qrels
data_files:
- split: qrels
path: qrels.jsonl
- config_name: corpus
data_files:
- split: corpus
path: corpus.jsonl
- config_name: query
data_files:
- split: query
path: query.jsonl
- config_name: hyde
data_files:
- split: gpt4
path: hyde/gpt4/query_with_hydoc.jsonl
- split: huatuo_o1_70b
path: hyde/huatuo-o1-70b/query_with_hydoc.jsonl
- split: llama_70b
path: hyde/llama-70b/query_with_hydoc.jsonl
- split: o3_mini
path: hyde/o3-mini/query_with_hydoc.jsonl
- split: qwen_7b
path: hyde/qwen-7b/query_with_hydoc.jsonl
- split: qwen_32b
path: hyde/qwen-32b/query_with_hydoc.jsonl
- split: qwen_72b
path: hyde/qwen-72b/query_with_hydoc.jsonl
- split: qwq_32b
path: hyde/qwq-32b/query_with_hydoc.jsonl
- split: r1_llama_70b
path: hyde/r1-llama-70b/query_with_hydoc.jsonl
- split: r1_qwen_32b
path: hyde/r1-qwen-32b/query_with_hydoc.jsonl
- config_name: query2doc
data_files:
- split: gpt4
path: query2doc/gpt4/query_with_hydoc.jsonl
- split: qwen_7b
path: query2doc/qwen-7b/query_with_hydoc.jsonl
- split: qwen_72b
path: query2doc/qwen-72b/query_with_hydoc.jsonl
- config_name: lamer
data_files:
- split: gpt4
path: lamer/gpt4/query_with_hydoc.jsonl
- split: qwen_7b
path: lamer/qwen-7b/query_with_hydoc.jsonl
- split: qwen_72b
path: lamer/qwen-72b/query_with_hydoc.jsonl
- config_name: search-r1
data_files:
- split: qwen_3b_ins
path: search-r1/qwen-3b-ins/query_with_hydoc.jsonl
- split: qwen_7b_ins
path: search-r1/qwen-7b-ins/query_with_hydoc.jsonl
- config_name: search-o1
data_files:
- split: qwq_32b
path: search-o1/qwq-32b/query_with_hydoc.jsonl
- split: qwen3_32b
path: search-o1/qwen3-32b/query_with_hydoc.jsonl
---

## 🔭 Overview

### R2MED: First Reasoning-Driven Medical Retrieval Benchmark

**R2MED** is a high-quality, high-resolution synthetic information retrieval (IR) dataset designed for medical scenarios. It contains 876 queries with three retrieval tasks, five medical scenarios, and twelve body systems.

| Dataset | #Q | #D | Avg. Pos | Q-Len | D-Len |
|:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|
| [Biology](https://huggingface.co/datasets/R2MED/Biology) | 103 | 57359 | 3.6 | 115.2 | 83.6 |
| [Bioinformatics](https://huggingface.co/datasets/R2MED/Bioinformatics) | 77 | 47473 | 2.9 | 273.8 | 150.5 |
| [Medical Sciences](https://huggingface.co/datasets/R2MED/Medical-Sciences) | 88 | 34810 | 2.8 | 107.1 | 122.7 |
| [MedXpertQA-Exam](https://huggingface.co/datasets/R2MED/MedXpertQA-Exam) | 97 | 61379 | 3.0 | 233.2 | 154.9 |
| [MedQA-Diag](https://huggingface.co/datasets/R2MED/MedQA-Diag) | 118 | 56250 | 4.4 | 167.8 | 179.7 |
| [PMC-Treatment](https://huggingface.co/datasets/R2MED/PMC-Treatment) | 150 | 28954 | 2.1 | 449.3 | 149.3 |
| [PMC-Clinical](https://huggingface.co/datasets/R2MED/PMC-Clinical) | 114 | 60406 | 2.2 | 182.8 | 480.4 |
| [IIYi-Clinical](https://huggingface.co/datasets/R2MED/IIYi-Clinical) | 129 | 10449 | 3.5 | 602.3 | 1273.0 |

## 🏆 Leaderboard
You could check out the results at [Leaderboard](https://r2med.github.io/#leaderboard).

## 🔗 GitHub
Github link [R2MED](https://github.com/R2MED/R2MED)

## 🏠 Homepage
Homepage link [R2MED](https://r2med.github.io/)

## 📄 Paper
Paper link [arXiv](https://arxiv.org/abs/2505.14558)