遇见数据集

AgenticGenomics: pharmacogenomics benchmark dataset for "Trustworthy agentic genomics requires validated skills, not better models" (matched five-cell, real-genome and seven-model input-normalisation analyses)

收藏
Zenodo2026-08-01 更新2026-08-01 收录
官方服务:

资源简介:

Benchmark dataset for Corpas et al. (2026), "Trustworthy agentic genomics requires validated skills, not better models" (under revision at Cell Genomics; CELL-GENOMICS-D-26-00551). Design. The synchronized package contains the matched five-cell comparison (13,200 attempted evaluations across eight models), the rule-extraction and bidirectional corrupted-contract experiments, GeT-RM external consensus comparisons, four-cohort real-genome analyses, and the input-normalisation experiment. Version 1.4.0 (1 August 2026). This version supersedes v1.3.1 for the input-normalisation result. Correction C14 makes an optional gene prefix presentation-neutral. Correction C15 accepts an explicit DIPLOTYPE marker inside Markdown, LaTeX or prose wrapping, applies the same policy to every model, and uses the last explicit marker as the final answer. Re-parsing 9,557 stored rows without new API calls recovered 87 marked calls and removed two provisional calls followed by an explicit final ABSTAIN. No raw response text was changed. Seven-model freeze. The definition-supplied arm contains 3,689 attempts: 2,905 calls, 780 abstentions and four output-budget truncations. The frozen analysis records source-file hashes, exact paired units and 10,000-replicate sample-cluster bootstrap intervals. For Claude Opus 4.5, the difference from the deterministic caller against GeT-RM is +0.010 with a 95% interval from -0.004 to +0.024; neither equivalence nor superiority is claimed. Gemini is assigned no performance estimate because a complete comparable run was not collected; its boxed pilot response is accepted by the same model-neutral parser. Reproducibility. Code is pinned to commit a367159 and immutable tag cg-revision-2026-08-01. CHECKSUMS.sha256 covers the corrected data, release notes, provenance record and tagged source archive. PROVENANCE.md states which per-call fields survive and explicitly discloses that wall-clock timestamps and provider response IDs were not captured. Cohort data-access and redistributability limitations are documented in the repository; no new sequencing was performed.

提供机构:
Zenodo
创建时间:
2026-08-01
二维码
社区交流群
二维码
科研交流群
商业服务