Radiology Report Generation Models Evaluation Dataset For Chest X-rays (RadEvalX)

The Radiology Report Generation Models Evaluation Dataset For Chest X-rays
(RadEvalX) is publicly available and developed similarly to the ReXVal
dataset. Just like ReXVal, RadEvalX focuses on radiologist evaluations of
errors found in automatically generated radiology reports. The dataset
includes annotations from two board-certified radiologists, who identified
clinically significant and clinically insignificant errors across eight
different categories of errors. Compared to the ground-truth reports from the
IU-Xray dataset, the evaluations were done on candidate radiology reports. For
every 100 studies and corresponding ground-truth reports, the dataset contains
one report generated using the M2Tr model from the corresponding X-ray image.
The radiologists then annotated these reports. The primary purpose of this
dataset is to assess the correlation between automated metrics and human
radiologists' scoring, explore the limitations of automated metrics, and
develop a model-based automated metric. This dataset has been created to
support further research in medical artificial intelligence (AI), particularly
in the field of radiology.

胸部X线影像放射报告生成模型评估数据集（RadEvalX）公开可获取，其构建思路与ReXVal数据集类似。与ReXVal一致，RadEvalX的核心聚焦于放射科医师对自动生成的放射报告中存在的错误开展评估。该数据集包含两位认证执业放射科医师的标注结果，他们从8类不同的错误类别中甄别出具有临床意义与无临床意义的错误。相较于IU-Xray数据集的标准真值报告，本次评估的对象为候选放射报告。每100项影像研究及其对应的标准真值报告配套下，数据集均包含1份由M2Tr模型基于对应X线影像生成的报告。随后由放射科医师对这些报告进行标注。本数据集的核心用途为：评估自动化评估指标与人类放射科医师评分之间的相关性，探究自动化评估指标的局限性，并研发基于模型的自动化评估指标。本数据集的构建旨在为医学人工智能（AI）领域，尤其是放射学方向的后续研究提供支撑。

搜集汇总

背景与挑战

背景概述

RadEvalX是一个用于评估胸部X光放射学报告生成模型的公开数据集，包含放射科医生对自动生成报告中错误的注释，旨在研究自动指标与人工评分的相关性及自动指标的局限性。