benchgap: LLM benchmark score snapshot of 8 October 2026 and reproduction package for "Mind the Gap"
收藏资源简介:
Reproduction package for the paper Mind the Gap: Explicit Cross-Benchmark Calibrations for Estimating Missing LLM Evaluation Scores (Süli, Eigner, Hollós). benchgap estimates missing large language model benchmark scores from explicit per-pair calibrations between benchmarks of the same capability. The live system (benchgap.net) refreshes daily and cannot regenerate a past state; this record freezes the state the paper analyses: the measured data retrieved on 8 October 2026, refitted end to end with the pipeline at commit 920b1c4. Contents: the SQLite snapshot (520 models, 359 benchmark versions, 6,180 measured scores with source and retrieval date, 14,314 estimates with provenance, 1,147 univariate calibrations, 21 multivariate mappings); the harness-tax analysis and its exports; the masked-holdout results (three random seeds and two sparse settings); the scripts that produce the paper's statistics, holdout evaluation and figures; and the pipeline source at commit 920b1c4. README.md gives the reproduction steps; SHA256SUMS lists checksums. Licences: the measured scores come from the public BenchLM.ai compilation (CC BY-NC 4.0), so the data files are released under CC BY-NC 4.0. The code (benchgap-920b1c4.zip and the three scripts) is MIT-licensed.



