ReXPref-Prior: A MIMIC-CXR Preference Dataset for Reducing Hallucinated Prior Exams in Radiology Report Generation

Generative vision-language models have exciting potential implications for radiology report generation, but unfortunately such models are also known to produce hallucinations and other nonsensical statements. For example, radiology report generation models regularly hallucinate prior exams, making statements such as “The lungs are hyperinflated with emphysematous changes as seen on prior CT” despite not having access to any prior exam. To address this shortcoming, we propose ReXPref-Prior, an adapted version of MIMIC-CXR where GPT-4 has removed references to prior exams from both findings and impression sections of chest X-ray reports. We expect ReXPref-Prior will be useful for training models that hallucinate prior exams less frequently, through techniques such as direct preference optimization. Additionally, ReXPref-Prior’s validation and test sets can be used as a new benchmark for evaluating report generation models.

生成式视觉-语言模型在放射学报告生成方面具有令人兴奋的潜在应用前景，但遗憾的是，此类模型也常常产生幻觉和其他不合逻辑的陈述。例如，放射学报告生成模型通常会虚构既往检查结果，如“肺过度膨胀，伴有肺气肿改变，如既往CT所见”，尽管它们并未访问到任何既往检查资料。为解决这一不足，我们提出了ReXPref-Prior，这是对MIMIC-CXR的一种改编版本，其中GPT-4已从胸部X光报告的发现和印象部分移除了对既往检查的引用。我们预计ReXPref-Prior将有助于训练那些较少虚构既往检查结果的模型，通过直接偏好优化等技术的应用。此外，ReXPref-Prior的验证集和测试集也可用作评估报告生成模型的新基准。