Air-gapped OpenScientist vs Exomiser on post-cutoff rare-disease cases: inputs, outputs and answer key
收藏资源简介:
Version 2 (2026-09-25): the strata were redefined. A case's causative gene is now a known disease gene if its gene–disease pair was annotated in HPOA v2025-08-11 at the cutoff or a publication before the cutoff had reported a patient with a germline variant in the gene, and novel if neither (primary-analysis set: known 75 = 46 by annotation + 29 by publication alone; novel 30; version 1 had 46 / 59 by annotation alone). The repository snapshot (commit 87a2a95) carries the per-case strata with their evidence (docs/case_strata.csv, docs/novel_gene_literature.csv, analysis/novel_genes/literature/) and the re-scored results, figures and tables; the run archives are unchanged. See README.md. Inputs, outputs and answer key for a benchmark comparing two arms on rare-disease cases from phenopacket-store first published on or after 2025-09-01: air-gapped OpenScientist (Claude Opus 4.6, no internet, a local PubMed mirror pruned to articles first public before 2025-09-01, able to run Exomiser) versus Exomiser 14.1.0 alone (data release 2508, stock exome preset). 133 primary-tier cases were run in both arms. 124 extension-tier cases were run in the Exomiser arm only. Each case is a sanitized phenopacket plus a simulated VCF with the causal variant spiked into a 1000 Genomes background exome. As a leakage control, an LLM-only probe (Opus 4.6, no tools) was run on the 133 post-cutoff cases and on 105 pre-cutoff control cases. Files: OpenScientist outputs (differentials, reports, transcripts, the agent's Exomiser runs), Exomiser-alone outputs including full exomiser.json, a SpliceAI sensitivity check, LLM probe outputs, the case VCFs, and a snapshot of the benchmark repository (code, cohort, scoring, paper). The answer key is inside the repository snapshot (cohort/answers.csv, paper/data/study_phenopackets/). See README.md.



