LATTE-CXR: Locally Aligned TexT and imagE, Explainable dataset for Chest X-Rays

Local annotation of medical data is both expensive and time-consuming due to the high cost of expert annotators, the precision required for accurate annotation, and the inherent challenges of medical diagnosis. To address these problems, we developed LATTE-CXR, a chest X-ray dataset with locally aligned image-text pairs, derived from the REFLACX dataset. LATTE-CXR supports tasks requiring local image-text annotations, such as phrase grounding, caption- guided object detection, and image captioning with region-level descriptions. By extracting statements from radiology reports corresponding to REFLACX annotated abnormalities, this dataset includes 3926 bounding box-statement pairs (with repeating statements) from 1668 MIMIC-CXR image readings in the REFLACX dataset. Additionally, we automatically generated 13751 bounding box- sentence pairs from 2,742 chest X-ray readings, utilizing timestamped eye- tracking data and transcribed reports from REFLACX. The eye-tracking bounding boxes are linked to corresponding annotated bounding boxes if they share a sentence, providing a comprehensive framework for assessing model explainability.由于专家标注人员成本高昂、精准标注所需的高精度要求，以及医学诊断本身固有的诸多挑战，医学数据的本地标注既成本高昂又耗时耗力。为解决上述问题，我们构建了LATTE-CXR数据集——这是一个源自REFLACX数据集、具备本地对齐图文对的胸部X射线（Chest X-ray）数据集。LATTE-CXR可支持需要本地图文标注的各类任务，例如短语定位（phrase grounding）、字幕引导的目标检测，以及带有区域级描述的图像字幕生成。通过从与REFLACX标注的异常区域对应的放射科报告（radiology report）中提取描述语句，本数据集从REFLACX数据集中的1668份MIMIC-CXR影像阅片结果中，获取了3926个边界框（bounding box）-语句对（含重复语句）。此外，我们利用REFLACX数据集带时间戳的眼动追踪数据（eye-tracking data）与转录报告，从2742份胸部X射线影像阅片结果中自动生成了13751个边界框-语句对。若眼动追踪边界框与标注边界框共享对应语句，则将二者进行关联，由此为模型可解释性（model explainability）评估提供了一套完整的研究框架。

搜集汇总

背景与挑战

背景概述

LATTE-CXR是一个胸部X光图像与文本局部对齐的可解释数据集，基于REFLACX数据集构建，包含3926个放射科医生标注的边界框-语句对和13751个自动生成的眼动追踪边界框-句子对。该数据集支持短语定位、图像描述等医学影像AI任务，并提供眼动追踪数据作为模型可解释性评估的基准。