Benchmark Evaluation of Large Language Models for Drug Combination Alerts

The benchmark study assessed 10 state-of-the-art large language models on an expert-curated set of 95 clinically risk drug combinations supported by 1,482 publications. All assessments used CoMed, an open-source, interpretable framework (GitHub: https://github.com/studentiz/comed; PyPI: https://pypi.org/project/comed/). CoMed prespecifies three configurations to attribute incremental value: native models without external evidence; retrieval-augmented generation that draws on PubMed, PubMed Central, and DailyMed; and clinician-informed chain-of-thought reasoning. To demonstrate the end-to-end process, the aspirin and warfarin case applied multi-agent adjudication across five predefined clinical dimensions and generated a structured HTML report with explicit source citations that synthesizes the combination’s risk for clinical interpretation.