<p>ANOVA results for accuracy.</p>

Artificial Intelligence (AI) has been dramatically applied to healthcare in various tasks to support clinicians in disease diagnosis and prognosis. It has been known that accurate diagnosis must be drawn from multiple evidence, namely clinical records, X-Ray images, IoT data, etc called the multi-modal data. Despite the existence of various approaches for multi-modal medical data fusion, the development of comprehensive systems capable of integrating data from multiple sources and modalities remains a considerable challenge. Besides, many machine learning models face difficulties in representation and computation due to the uncertainty and diversity of medical data. This study proposes a novel multi-modal fuzzy knowledge graph framework, called FKG-MM, which integrates multi-modal medical data from multiple sources, offering enhanced computational performance compared to unimodal data. In addition, the FKG-MM framework is based on the fuzzy knowledge graph model, one of the models that represent and compute effectively with medical data in tabular form. Through some experiment scenarios utilizing the well-known BRSET dataset on multi-modal diabetic retinopathy, it has been experimentally validated that the feature selection method, when combining image features with tabular medical data features, gives the highest reliability results among 5 methods including Feature Selection Method, Tensor Product, Hadamard Product, Filter Selection, and Wrapper Selection. In addition, the experiment also confirms that the accuracy of FKG-MM increases by 12–14% when combining image data with tabular medical data than the related methods diagnosing only on tabular data.

人工智能（Artificial Intelligence, AI）已被广泛且深度地应用于医疗领域的多项任务，助力临床医师开展疾病诊断与预后工作。众所周知，精准诊断必须依托多源证据，即临床病历、X射线影像、物联网（Internet of Things, IoT）数据等，此类数据统称为多模态数据（multi-modal data）。尽管目前已存在多种多模态医疗数据融合方法，但构建能够整合多源多模态数据的综合系统仍面临不小的挑战。此外，由于医疗数据兼具不确定性与多样性，诸多机器学习模型在数据表征与计算过程中均存在困难。本研究提出一种新颖的多模态模糊知识图谱框架，命名为FKG-MM，该框架可整合多源多模态医疗数据，相较于单模态数据，能够实现更优异的计算性能。此外，FKG-MM框架基于模糊知识图谱模型，该类模型可有效表征并计算表格形式的医疗数据。通过采用知名的多模态糖尿病视网膜病变BRSET数据集开展多组实验场景，实验结果证实，在特征选择方法、张量积（Tensor Product）、阿达马积（Hadamard Product）、过滤式选择（Filter Selection）与包装式选择（Wrapper Selection）这5种方法中，将图像特征与表格医疗数据特征相结合的特征选择方法取得了最高的可靠性表现。此外，实验还证实，相较于仅基于表格数据进行诊断的同类方法，FKG-MM结合图像数据与表格医疗数据时，准确率提升了12%至14%。