disi-unibo-nlp/ReMedQA

ReMedQA数据集旨在解决医学多选题回答（MCQA）基准测试中的局限性，通过引入开放式答案变体和系统性扰动项来扩展标准的医学MCQA数据集（MedQA、MedMCQA和MMLU-med）。该数据集设计了两种可靠性导向的指标（ReAcc和ReCon），分别衡量在所有变体中正确回答问题的比例和答案的一致性。数据集包含多个子集（medqa、medmcqa、mmlu），每个子集都有多种扰动模式（如mcq、open、incorrect等）。数据集的特征包括id、question、options、answer、prompt和prompt_think。

The ReMedQA dataset addresses limitations in medical multiple-choice question answering (MCQA) benchmarks by extending standard medical MCQA datasets—MedQA, MedMCQA, and MMLU-med—with open-answer variants and systematically perturbed items. It introduces ReAcc and ReCon, two reliability-oriented metrics that measure the proportion of correctly answered questions across variations and answer consistency, respectively. The dataset is organized into subsets (medqa, medmcqa, mmlu) with various perturbation modes (e.g., mcq, open, incorrect). Features include id, question, options, answer, prompt, and prompt_think.