Radiology Report Expert Evaluation (ReXVal) Dataset

The Radiology Report Expert Evaluation (ReXVal) Dataset is a publicly available dataset of radiologist evaluations of errors in automatically generated radiology reports. The dataset contains annotations from 6 board certified radiologists on clinically significant and clinically insignificant errors under 6 error categories for candidate radiology reports with respect to ground-truth reports from the MIMIC-CXR dataset. There are 4 candidate reports generated for 50 studies, translating to 200 pairs of candidate and ground-truth reports on which radiologists provided annotations. The dataset has been used to evaluate the alignment between scoring of automated metrics and that of radiologists, investigate the failure modes of automated metrics, and build a composite automated metric, in a study on how to meaningfully measure progress in radiology report generation. It is also created to support additional medical AI research in radiology and other expert tasks.

《放射学报告专家评估（ReXVal）数据集》系一项公开数据集，收录了放射学家对自动生成放射学报告中错误进行的评估。该数据集包含6位具备执业资格的放射学家对候选放射学报告中6类临床意义及临床非意义错误进行的标注，这些候选报告与MIMIC-CXR数据集的真实报告进行比对。针对50项研究，共生成4份候选报告，形成200对候选报告与真实报告的标注对。该数据集已被应用于评估自动评分指标与放射学家评分的一致性，探究自动评分指标的失效模式，并在一项关于如何有意义地衡量放射学报告生成进步的研究中构建了综合自动化评分指标。此外，该数据集的创建旨在支持放射学及其他专家任务领域的额外医学人工智能研究。