RAG vs. No-RAG benchmark dataset: LLM-judged clinical reasoning in transplant infectious disease
收藏资源简介:
Dataset supporting the study "Retrieval-Augmented Generation Reduces Synthesis-Intensive Clinical Reasoning Quality in Transplant Infectious Disease." Contains three files: all_models_rag_non_rag_judged.csv — Primary dataset. 4,911 LLM-judge records from 1,637 unique model outputs (39 models × 21 transplant infectious disease clinical vignettes × 2 conditions [RAG / No-RAG], scored by 3 independent LLM judges on 6 clinical reasoning dimensions). Includes generation-time grounding metrics and major-error annotations. expert_evaluations_linked_long.csv — Blinded domain-expert evaluations. 60 ratings from 3 experts across 10 paired RAG/No-RAG cases (accuracy, usefulness, preference). retrieval_quality_classification.csv — Per-case retrieval quality audit. Document-level classification of all 21 cases into 6 retrieval quality categories with similarity scores and per-case RAG-benefit deltas. Analysis code and figure generation scripts: https://github.com/KeatingLabNYU/rag-txid-benchmark



