Benchmarking large language models for drug combination alerts: achieving expert-level reliability via knowledge grounding and contextual reasoning

This study systematically evaluated the potential of LLMs for drug combination alerting, focusing on four key aspects: (1) the baseline performance of native LLMs, (2) the contribution of external knowledge grounding via Retrieval-Augmented Generation (RAG), (3) the impact of expert-guided reasoning using context engineering, and (4) the utility of a multi-agent architecture for comprehensive, interpretable risk analysis.

The source code used to conduct all analyses in this study has been deposited in a public GitHub repository (https://github.com/studentiz/comed) and is available as a Python package (https://pypi.org/project/comed/).

本研究系统评估了大语言模型（LLM）在药物联用预警任务中的应用潜力，重点聚焦四大核心维度：（1）原生大语言模型的基准性能；（2）借助检索增强生成（Retrieval-Augmented Generation，RAG）实现的外部知识增强效果；（3）采用上下文工程的专家引导推理对模型性能的影响；（4）多智能体架构用于开展全面且可解释的风险分析的实用价值。本研究所有分析所用的源代码已存入公开GitHub仓库（https://github.com/studentiz/comed），同时可作为Python软件包（https://pypi.org/project/comed/）获取使用。