M3KG
收藏资源简介：
M3KG是一个基于真实医学报告构建的大规模多模态医学知识图谱，包含2477个实体、3种关系、37424个三元组和6943个疾病感知视觉Tokens，用于CheXpert Plus数据集。它通过GPT-4o生成训练数据，并使用ReXKG进行实体和关系提取，构建了一个包含解剖结构、疾病、概念、设备、程序、尺寸和视觉Tokens等六种实体类型和修改、位于、提示三种关系类型的知识图谱。该数据集旨在为基于大型语言模型的放射学报告生成提供大规模知识图谱，解决现有医学知识图谱规模有限、多模态信息缺失和静态图谱无法支持多级知识关联等问题。
M3KG is a large-scale multimodal medical knowledge graph constructed from real-world medical reports. It contains 2477 entities, 3 relationship types, 37424 triples, and 6943 disease-aware visual Tokens, and is tailored for the CheXpert Plus dataset. It generates training data via GPT-4o and uses ReXKG to perform entity and relation extraction, building a knowledge graph that covers six entity types: anatomical structures, diseases, concepts, devices, procedures, dimensions, and visual Tokens, as well as three relationship types: modification, located in, and indication. This dataset is designed to provide large-scale knowledge graphs for radiology report generation based on large language models, addressing the limitations of existing medical knowledge graphs, including limited scale, lack of multimodal information, and the inability of static knowledge graphs to support multi-level knowledge association.
数据集概述
数据集基本信息
- 名称: Medical_Image_Analysis
- 领域: 医学影像分析与报告生成
- 主要应用: X光影像报告生成、疾病识别、医学影像预训练
主要项目与论文
R2GenKG: 基于多模态知识图谱的放射学报告生成
- 论文: R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation
- 内容: 构建大规模多模态医学知识图谱（M3KG），包含2477个实体、3种关系、37424个三元组和6943个疾病感知视觉标记。采用Swin-Transformer提取视觉特征，结合知识图谱和疾病感知视觉标记生成报告。
疾病感知视觉标记记忆激活的X光报告生成
- 论文: Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generation
- 内容: 提出一种新型关联记忆增强模型，通过Hopfield网络建立疾病相关标记的记忆关联，生成高质量医学报告。
CXPMRG-Bench: CheXpert Plus数据集上的预训练与基准测试
- 论文: CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Dataset
- 内容: 在CheXpert Plus数据集上对主流X光报告生成模型和大语言模型进行全面基准测试，提出多阶段预训练策略。
R2GenCSR: 基于上下文样本检索的X光报告生成
- 论文: R2GenCSR: Retrieving Context Samples for Large Language Model based X-ray Medical Report Generation
- 内容: 引入Mamba作为视觉骨干网络，结合上下文检索增强特征表示，生成高质量医学报告。
高分辨率X光影像预训练研究
- 论文: Pre-training on High Definition X-ray Images: An Experimental Study
- 内容: 提出首个高分辨率（1280×1280）X光预训练基础视觉模型，采用上下文感知掩码策略。
数据集与基准
- CheXpert Plus数据集: 用于X光报告生成和疾病识别的基准数据集。
- 其他数据集: IU-Xray、MIMIC-CXR。
相关资源
引用
bibtex @misc{wang2025r2genKG, title={R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation}, author={Futian Wang and Yuhan Qiao and Xiao Wang and Fuling Wang and Yuxiang Zhang and Dengdi Sun}, year={2025}, eprint={2508.03426}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2508.03426}, }
