AgenticGenomics: pharmacogenomics benchmark dataset for 'Validated skills are necessary but not sufficient for trustworthy agentic genomics'
收藏资源简介:
Benchmark dataset for Corpas et al. (2026), "Trustworthy agentic genomics requires validated skills, not better models" (under revision at Cell Genomics; CELL-GENOMICS-D-26-00551). Design. The synchronized package contains the matched five-cell comparison (13,200 attempted evaluations across eight models), the rule-extraction and bidirectional corrupted-contract experiments, GeT-RM external consensus comparisons, four-cohort real-genome analyses, and the input-normalisation experiment. Version 1.4.1 (1 August 2026). This version supersedes v1.4.0. During Zenodo version import, nine same-named pre-C15 files were retained instead of replaced. A public-side checksum audit detected the mismatch. Version 1.4.1 explicitly replaces those files and verifies every staged release file byte-for-byte against the live record. Parser corrections. C14 makes an optional gene prefix presentation-neutral. C15 accepts an explicit DIPLOTYPE marker inside Markdown, LaTeX or prose wrapping, applies the same policy to every model, and uses the last explicit marker as the final answer. Re-parsing 9,557 stored rows without new API calls recovered 87 marked calls and removed two provisional calls followed by an explicit final ABSTAIN. No raw response text was changed. Seven-model freeze. The definition-supplied arm contains 3,689 attempts: 2,905 calls, 780 abstentions and four output-budget truncations. The frozen analysis records source-file hashes, exact paired units and 10,000-replicate sample-cluster bootstrap intervals. For Claude Opus 4.5, the difference from the deterministic caller against GeT-RM is +0.010 with a 95% interval from -0.004 to +0.024; neither equivalence nor superiority is claimed. Gemini is assigned no performance estimate because a complete comparable run was not collected; its boxed pilot response is accepted by the same model-neutral parser. Reproducibility. Code is pinned to commit aeabc20 and immutable tag cg-revision-2026-08-01b. CHECKSUMS.sha256 covers the corrected data, release notes, provenance record and tagged source archive. PROVENANCE.md states which per-call fields survive and explicitly discloses that wall-clock timestamps and provider response IDs were not captured. Cohort data-access and redistributability limitations are documented in the repository; no new sequencing was performed.



