PAHD benchmarks: screening and extraction

These are the two benchmarks used to evaluate the large language model (LLM) pipeline that built PAHD (Physical Activity for Hypertension Database). Each file holds the adjudicated benchmark labels and the pipeline's output for the same items, side by side, with the result of each comparison. Every accuracy figure of the screening and extraction evaluation in the accompanying article (Tables 2 and 3, Figures 3 and 4) can be recomputed from these two files. Each benchmark was annotated independently by two annotators, and a human adjudicator resolved every disagreement against the paper. The annotators' own answers are not included; the files hold only the final (adjudicated) labels. Screening benchmark: 500 of the 2,548 records that entered LLM screening. They were sampled in three strata by keyword flags in the title, abstract and MeSH terms: S1, hypertension and exercise terms: 225 of 880 S2, exercise term only: 200 of 920 S3, no exercise term: 75 of 748 Each record was judged against the article's five eligibility criteria; 59 records are eligible. Each record carries a sampling weight, so that weighted estimates describe all 2,548 records. Extraction benchmark: 49 of the 255 included studies were sampled by type of exercise. Of these, 36 form the test set: they were held out from prompt development, and all extraction results of the article are reported on them. The other 13 studies, the development set, are not included. For BP outcomes, results were adjudicated at the level of single data points. Files: screening_benchmark.csv: 500 PubMed records, one per row. Each row has the stratum and sampling weight, the adjudicated answers to criteria 1–5, the final eligibility decision, and the pipeline's answers in the title-and-abstract and full-text steps. extraction_benchmark.csv: the 36 test studies, with 4,586 scored items: arms, interventions and arm–intervention links time points and outcomes study, arm and intervention fields FITT parameters protocol sentences baseline statistics BP data points Each item has the benchmark value, the pipeline value and the result of the comparison. README.md: column definitions, the scoring rules for each item type, and how to reproduce the reported metrics Database: PAHD, https://doi.org/10.5281/zenodo.23574404 Please cite: Li C, Obioha C, Cui X, Zhou X, Rosenzweig A, Wang M. A Knowledge Database of Physical Activity Interventions for Hypertension using Large Language Models. Manuscript submitted for publication.