MIMIC-Eye: Integrating MIMIC Datasets with REFLACX and Eye Gaze for Multimodal Deep Learning Applications
收藏资源简介：
Deep learning technologies have been widely adopted in medical imaging due to their ability to extract features from images and make accurate diagnoses automatically. Medical imaging technologies are particularly useful because they can be trained to detect subtle differences in images that are hard to detect for human radiologists. In the real world, radiologists must rely on various types of patient information to assess medical images confidently. However, most DL applications in medical imaging only utilize image data, mainly because the literature on medical datasets combining different data modalities is scarce. In this study, we present MIMIC-EYE, a dataset that encompasses a comprehensive integration of several datasets related to MIMIC. This dataset includes a comprehensive range of patient information, including medical images and reports (MIMIC CXR and MIMIC JPG), clinical data (MIMIC IV ED), a detailed account of the patient's hospital journey (MIMIC IV), and eye tracking data containing gaze information and pupil dilations together with image annotations (REFLACX and EYE GAZE). Integrating eye tracking data with the various MIMIC modalities may provide a more comprehensive understanding of radiologists' visual search behavior patterns and facilitate the development of more robust, accurate, and reproducible deep-learning models for medical imaging diagnosis.
深度学习技术在医学影像领域的应用已得到广泛推广，得益于其从图像中提取特征并实现自动诊断的能力。医学影像技术尤为适用，因其可被训练以探测图像中难以被人类放射科医生察觉的细微差异。在现实世界中，放射科医生必须依赖多种患者信息来自信地评估医学影像。然而，大多数深度学习在医学影像中的应用仅利用图像数据，主要原因是关于结合不同数据模态的医学数据集的文献稀缺。在本研究中，我们提出了MIMIC-EYE数据集，该数据集综合了与MIMIC相关的多个数据集。该数据集包含了一系列全面的病人信息，包括医学图像和报告（MIMIC CXR和MIMIC JPG）、临床数据（MIMIC IV ED）、患者住院过程的详细记录（MIMIC IV），以及包含注视信息、瞳孔扩张和图像标注的眼动数据（REFLACX和EYE GAZE）。将眼动数据与MIMIC的多种模态相结合，可能有助于更全面地理解放射科医生的视觉搜索行为模式，并促进开发更为稳健、精确且可重复的医学影像诊断深度学习模型。
