Radiology Report Generation Models Evaluation Dataset For Chest X-rays (RadEvalX)

The Radiology Report Generation Models Evaluation Dataset For Chest X-rays (RadEvalX) is publicly available and developed similarly to the ReXVal dataset. Just like ReXVal, RadEvalX focuses on radiologist evaluations of errors found in automatically generated radiology reports. The dataset includes annotations from two board-certified radiologists, who identified clinically significant and clinically insignificant errors across eight different categories of errors. Compared to the ground-truth reports from the IU-Xray dataset, the evaluations were done on candidate radiology reports. For every 100 studies and corresponding ground-truth reports, the dataset contains one report generated using the M2Tr model from the corresponding X-ray image. The radiologists then annotated these reports. The primary purpose of this dataset is to assess the correlation between automated metrics and human radiologists' scoring, explore the limitations of automated metrics, and develop a model-based automated metric. This dataset has been created to support further research in medical artificial intelligence (AI), particularly in the field of radiology.

该胸部X光片放射学报告生成模型评估数据集（RadEvalX）已公开并提供，其开发方式与ReXVal数据集相似。与ReXVal一样，RadEvalX专注于对自动生成的放射学报告中发现的错误进行的放射科医师评估。该数据集包含了两位资深放射科医师的标注，他们识别了八种不同类别中的临床显著错误和临床非显著错误。与IU-Xray数据集的基准报告相比，评估是在候选放射学报告上进行的。对于每100项研究和相应的基准报告，该数据集包含一份由对应X光图像生成的M2Tr模型报告。随后，放射科医师对这些报告进行了标注。本数据集的主要目的是评估自动化指标与人类放射科医师评分之间的相关性，探讨自动化指标的限制，并开发基于模型的自动化指标。该数据集的创建旨在支持医疗人工智能（AI）领域的进一步研究，特别是放射学领域。