遇见数据集

val2026: blinded claim-level validation of data references extracted from 182 heliophysics papers

收藏
Zenodo2026-09-24 更新2026-10-01 收录
官方服务:

资源简介:

v3 adds the grounding catalog (403 observatories, 977 instruments) and an instrument_id column in claims.csv. Claims, verdicts and all numbers are unchanged from v2.Version 2 adds the second-pass adjudication records; see READMEVerdicts from a blinded two-reviewer validation campaign (August–September 2026) over every data reference — (paper, instrument, observation window) — extracted from a 182-paper heliophysics evaluation set by two configurations of the paper-data-linking pipeline (GPT-OSS-120B at mixed reasoning effort, and GPT-5.4). Contains the evaluation set with sampling strata and per-paper licenses, 1,613 deduplicated claims, 1,829 reviewer verdicts with checkboxes and notes, the calibration-claim list, the manual day-level adjudications, the archived analysis snapshots (including the pre-adjudication snapshot used for inter-rater statistics), the analysis and table-rendering scripts, the ratified review rubric (v2.1) and protocol, and run provenance for every configuration. Version 1 holds the first-pass, pre-adjudication verdicts; a second version will follow adjudication. Paper full texts are not redistributed; DOIs and text checksums are provided. Companion to the software record https://doi.org/10.5281/zenodo.22899457 and a paper in preparation.

提供机构:
Zenodo
创建时间:
2026-09-24
二维码
社区交流群
二维码
科研交流群
商业服务