MIMIC-Eye: Integrating MIMIC Datasets with REFLACX and Eye Gaze for Multimodal Deep Learning Applications
收藏资源简介：
Deep learning technologies have been widely adopted in medical imaging due to their ability to extract features from images and make accurate diagnoses automatically. Medical imaging technologies are particularly useful because they can be trained to detect subtle differences in images that are hard to detect for human radiologists. In the real world, radiologists must rely on various types of patient information to assess medical images confidently. However, most DL applications in medical imaging only utilize image data, mainly because the literature on medical datasets combining different data modalities is scarce. In this study, we present MIMIC-EYE, a dataset that encompasses a comprehensive integration of several datasets related to MIMIC. This dataset includes a comprehensive range of patient information, including medical images and reports (MIMIC CXR and MIMIC JPG), clinical data (MIMIC IV ED), a detailed account of the patient's hospital journey (MIMIC IV), and eye tracking data containing gaze information and pupil dilations together with image annotations (REFLACX and EYE GAZE). Integrating eye tracking data with the various MIMIC modalities may provide a more comprehensive understanding of radiologists' visual search behavior patterns and facilitate the development of more robust, accurate, and reproducible deep-learning models for medical imaging diagnosis.
深度学习技术已在医学影像领域得到广泛应用，因其可从图像中提取特征并自动完成精准诊断。医学影像技术的优势尤为突出，可经训练后检测出人类放射科医师难以察觉的图像细微差异。在实际临床场景中，放射科医师需依托各类患者信息才能自信地对医学影像进行评估。然而当前绝大多数深度学习（Deep Learning, DL）在医学影像领域的应用仅使用图像数据，这主要是因为整合多模态数据的医学数据集相关研究文献较为匮乏。本研究推出MIMIC-EYE数据集，该数据集全面整合了多个与MIMIC相关的数据集。此数据集涵盖多维度患者信息：包括医学影像与报告（MIMIC CXR、MIMIC JPG）、临床数据（MIMIC IV ED）、患者完整住院历程详情（MIMIC IV），以及包含注视信息与瞳孔扩张数据的眼动追踪数据，同时附带图像标注信息（REFLACX与EYE GAZE）。将眼动追踪数据与各类MIMIC多模态数据相结合，能够更全面地解析放射科医师的视觉搜索行为模式，助力开发更稳健、精准且可复现的医学影像诊断深度学习模型。
