Coded data, codebook and code: reliability reporting for human ratings and LLM judges in AI mental-health research
收藏资源简介:
Coded data, codebook and analysis code for a design-weighted audit of how LLM-heavy AI mental-health papers (2023 to 2026) report human ratings and LLM-judge validation. Contains the 20-field codebook, the 200-paper year-stratified coded sample with short evidence quotes and design weights, a log of the six codes corrected during the author audit, the raw and corrected reader codes, the 120-paper calibration sample, the second blind reader pass (40 papers), and make_assets.py, which regenerates every number, table and figure of the companion manuscript. All codes were produced by LLM readers (Claude) and audited by the author; there is no human double-coding. Full texts of the source papers are not redistributed. Code licence MIT; data licence CC BY 4.0.



