LATTE-CXR: Locally Aligned TexT and imagE, Explainable dataset for Chest X-Rays

Local annotation of medical data is both expensive and time-consuming due to
the high cost of expert annotators, the precision required for accurate
annotation, and the inherent challenges of medical diagnosis. To address these
problems, we developed LATTE-CXR, a chest X-ray dataset with locally aligned
image-text pairs, derived from the REFLACX dataset. LATTE-CXR supports tasks
requiring local image-text annotations, such as phrase grounding, caption-
guided object detection, and image captioning with region-level descriptions.
By extracting statements from radiology reports corresponding to REFLACX
annotated abnormalities, this dataset includes 3926 bounding box-statement
pairs (with repeating statements) from 1668 MIMIC-CXR image readings in the
REFLACX dataset. Additionally, we automatically generated 13751 bounding box-
sentence pairs from 2,742 chest X-ray readings, utilizing timestamped eye-
tracking data and transcribed reports from REFLACX. The eye-tracking bounding
boxes are linked to corresponding annotated bounding boxes if they share a
sentence, providing a comprehensive framework for assessing model
explainability.