Dataset for the study: A Large Language Model Approach to Extracting Causal Evidence across Study Designs for Evidence Triangulation

Health strategies increasingly emphasize both behavioral and biomedical interventions, yet the complex and often contradictory guidance on diet, behavior, and health outcomes complicates evidence-based decision-making. Evidence triangulation across diverse study designs is essential for balancing biases and establishing causality, but scalable, automated methods for achieving this are lacking. In this study, we assess the performance of large language models (LLMs) in extracting both ontological and methodological information from scientific literature to automate evidence triangulation. A two-step extraction approach—focusing on factor-effect concepts first, followed by relation extraction—outperformed a one-step method, particularly in identifying effect direction and statistical significance. Using salt intake and blood pressure as a case study, we calculated the Convergeny of Evidence (CoE) and Level of Convergency (LoC), finding a strong excitatory effect of salt on blood pressure, and weak excitatory effect on cardiovascular diseases and deaths. This approach complements traditional meta-analyses by integrating evidence across study designs, also providing dynamic tracking of evidence on controversies.

健康战略日益同时重视行为干预与生物医学干预，但当前关于饮食、行为与健康结局的复杂且时常相互矛盾的指导意见，却为循证决策增添了复杂性。采用多样化研究设计开展证据三角验证（evidence triangulation），是平衡研究偏倚、确立因果关系的核心要求，但目前仍缺乏可规模化落地的自动化实现方法。本研究评估了大语言模型（large language models，LLMs）从科学文献中抽取本体论信息与方法论信息、以实现证据三角验证自动化的性能表现。我们提出的两步抽取法——先聚焦因子-效应概念，再开展关系抽取——其性能优于单步抽取法，尤其在识别效应方向与统计学显著性方面表现突出。本研究以盐摄入量与血压为案例，计算了证据收敛性（Convergeny of Evidence，CoE）与收敛等级（Level of Convergency，LoC），结果发现盐对血压具有显著的兴奋性效应，而对心血管疾病与死亡的兴奋性效应较弱。该方法通过整合不同研究设计的证据，可对传统荟萃分析形成有效补充，同时还能动态追踪争议性议题相关的证据进展。