MIMIC-CXR-Ext-ILS: Lesion Segmentation Masks and Instruction-Answer Pairs for Chest X-rays

The applicability of current lesion segmentation models for chest X-rays
(CXRs) has been limited both by a small number of target labels and the
reliance on complex, expert-level text inputs, creating a barrier to practical
use. To address these limitations, we introduce instruction-guided lesion
segmentation (ILS), a medical-domain adaptation of referring image
segmentation (RIS) designed to segment diverse lesion types based on simple,
user-friendly instructions. Under this task, we construct MIMIC-CXR-Ext-ILS,
the first large-scale instruction-answer dataset for CXR lesion segmentation,
using our fully automated multimodal pipeline that generates annotations from
CXR images and their corresponding reports. MIMIC-CXR-Ext-ILS contains 1.1M
instruction-answer pairs derived from 192K images and 91K unique segmentation
masks, covering seven major lesion types. Despite being constructed entirely
without human intervention, expert evaluations report a high acceptance rate
of over 95% for this dataset.