Medical_Multimodal_Evaluation_Data

## Evaluation Guide

This dataset is used to evaluate medical multimodal LLMs, as used in [HuatuoGPT-Vision](https://github.com/FreedomIntelligence/HuatuoGPT-Vision). It includes benchmarks such as `VQA-RAD`, `SLAKE`, `PathVQA`, `PMC-VQA`, `OmniMedVQA`, and `MMMU-Medical-Tracks`.

To get started:
1. **Download the dataset** and extract the `images.zip` file.
2. **Find evaluation code** on our GitHub: [HuatuoGPT-Vision](https://github.com/FreedomIntelligence/HuatuoGPT-Vision).

This open-source release aims to simplify the evaluation of medical multimodal capabilities in large models. Please cite the relevant benchmark papers in your work.

## Citation

If you find our data useful, please consider citing our work!
```
@misc{chen2024huatuogptvisioninjectingmedicalvisual,
title={HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale},
author={Junying Chen and Ruyi Ouyang and Anningzhe Gao and Shunian Chen and Guiming Hardy Chen and Xidong Wang and Ruifei Zhang and Zhenyang Cai and Ke Ji and Guangjun Yu and Xiang Wan and Benyou Wang},
year={2024},
eprint={2406.19280},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2406.19280},
}
```

# 评估指南

本数据集用于评估医疗多模态大语言模型（Large Language Model, LLM），适配[HuatuoGPT-Vision](https://github.com/FreedomIntelligence/HuatuoGPT-Vision)的使用场景。其涵盖的基准测试集包括`VQA-RAD`、`SLAKE`、`PathVQA`、`PMC-VQA`、`OmniMedVQA`以及`MMMU-Medical-Tracks`。

## 快速上手步骤
1. 下载本数据集并解压`images.zip`压缩包。
2. 前往我们的GitHub仓库[HuatuoGPT-Vision](https://github.com/FreedomIntelligence/HuatuoGPT-Vision)获取评估代码。

本次开源发布旨在简化大模型医疗多模态能力的评估流程，敬请在相关研究工作中引用对应基准测试集的原创论文。

## 引用说明
若本数据集对您的研究有所帮助，请引用我们的相关工作！

@misc{chen2024huatuogptvisioninjectingmedicalvisual,
title={HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale},
author={Junying Chen and Ruyi Ouyang and Anningzhe Gao and Shunian Chen and Guiming Hardy Chen and Xidong Wang and Ruifei Zhang and Zhenyang Cai and Ke Ji and Guangjun Yu and Xiang Wan and Benyou Wang},
year={2024},
eprint={2406.19280},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2406.19280},
}