Supplementary Material for: From development to implementation: a systematic review on the current maturity status of artificial intelligence models for patients with colorectal cancer liver metastases

Introduction Artificial intelligence (AI) is increasingly being researched and developed in the medical field and holds potential to transform healthcare after successful implementation. For patients with colorectal cancer liver metastases (CRLM), many AI models have been developed, but knowledge about translation of these models in the clinical workflow is lacking. Therefore, this systematic review aims to provide a contemporary overview of the current maturity status of AI models for patients with CRLM. Methods A systematic search of the literature until November 2, 2023 was conducted in PubMed, Embase.com and Clarivate Analytics/Web of Science Core Collection to identify eligible studies. Studies using AI and/or radiomics for patients with CRLM were considered eligible. Data on the study aim, study design, size of dataset, country, type of AI application, level of validation and clinical implementation status (NASA technology readiness levels) were collected. Risk of bias and applicability of the individual studies were evaluated using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). Results A total of 117 studies were included. Ninety-seven studies (83%) were published in the last five years. The most common study design was retrospective (96%). Thirty-five studies (30%) utilized a dataset of fewer than 50 patients with CRLM. Internal validation was performed in 63% of studies, external validation in 17%. The remaining studies did not report validation. Half of the studies were classified as high risk of bias. None of the included studies performed real-time testing, workflow integration, clinical testing or clinical integration. Conclusion Although a rapid increase in research describing the development of AI models for patients with CRLM is observed in recent years, not a single AI model has been translated into clinical practice.引言 人工智能（Artificial Intelligence，AI）在医疗领域的研究与开发日益深入，且在成功落地后具备变革医疗健康行业的潜力。针对结直肠癌肝转移（colorectal cancer liver metastases, CRLM）患者，目前已开发出多款人工智能模型，但关于这些模型在临床工作流中的转化应用的相关认知仍较为匮乏。因此，本系统综述旨在全面梳理当前针对CRLM患者的人工智能模型的成熟度现状。 方法 本研究于2023年11月2日前，在PubMed、Embase.com以及科睿唯安Web of Science核心合集（Clarivate Analytics/Web of Science Core Collection）中开展系统性文献检索，以筛选符合纳入标准的研究。凡针对CRLM患者使用人工智能和/或放射组学（radiomics）的研究均符合纳入条件。我们收集了以下数据：研究目的、研究设计、数据集规模、研究开展国家、人工智能应用类型、验证级别以及临床实施状态（采用美国国家航空航天局（National Aeronautics and Space Administration, NASA）技术就绪等级）。采用预测模型偏倚风险评估工具（Prediction model Risk Of Bias ASsessment Tool, PROBAST）对各项研究的偏倚风险与适用性进行评估。 结果 本研究共纳入117项研究。其中97项（83%）发表于近五年。最常见的研究设计为回顾性研究，占比达96%。35项研究（30%）使用的CRLM患者数据集规模不足50例。63%的研究开展了内部验证，17%开展了外部验证，剩余研究未报告验证相关内容。半数研究被归类为存在高偏倚风险。纳入的所有研究均未开展实时测试、工作流集成、临床测试或临床集成。 结论 尽管近年来针对CRLM患者开发人工智能模型的相关研究数量呈快速增长趋势，但目前尚无任何一款人工智能模型成功转化至临床实践中。