MS-CXR-T: Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing

MS-CXR-T is a multi-modal benchmark dataset for evaluating biomedical vision-
language processing (VLP) models on two distinct temporal tasks in radiology:
image classification and sentence similarity. The former comprises multi-image
frontal chest X-rays with ground-truth labels (N=1326) across 5 findings, with
classes corresponding to 3 states of disease progression for each finding:
{'Improving', 'Stable', 'Worsening'}, expanding on the Chest ImaGenome
progression dataset. The latter quantifies the temporal-semantic similarity of
text embeddings extracted from pairs of sentences (N=361). The pairs can be
either paraphrases or contradictions in terms of disease progression. The data
for both tasks was manually annotated and reviewed by a board-certified
radiologist. The dataset provides researchers an opportunity to evaluate both
image and text models on these biomedical temporal tasks and reproduce
experiments reported in the corresponding literature.