Radiology Report Expert Evaluation (ReXVal) Dataset

The Radiology Report Expert Evaluation (ReXVal) Dataset is a publicly available dataset of radiologist evaluations of errors in automatically generated radiology reports. The dataset contains annotations from 6 board certified radiologists on clinically significant and clinically insignificant errors under 6 error categories for candidate radiology reports with respect to ground-truth reports from the MIMIC-CXR dataset. There are 4 candidate reports generated for 50 studies, translating to 200 pairs of candidate and ground-truth reports on which radiologists provided annotations. The dataset has been used to evaluate the alignment between scoring of automated metrics and that of radiologists, investigate the failure modes of automated metrics, and build a composite automated metric, in a study on how to meaningfully measure progress in radiology report generation. It is also created to support additional medical AI research in radiology and other expert tasks.