BIG-Bench Hard (BBH) Data Quality Investigation Report
收藏Zenodo2025-12-30 更新2026-05-26 收录
下载链接：
https://zenodo.org/doi/10.5281/zenodo.18091404
下载链接
链接失效反馈官方服务：
资源简介：
This dataset documents 49 ground truth labeling errors discovered in the BIG-Bench Hard (BBH) benchmark through systematic multi-model cross-verification and mathematical proof.
v1.5.0 Updates
Added comprehensive 8-model verification analysis (GPT-5.2 Pro, Claude Opus 4.5, DeepSeek-Reasoner, Llama 3.3 70B, Gemini Pro 3)
Included mathematical SVG coordinate proofs for all 42 geometric shape errors
Documented Gemini discrepancy resolution through mathematical verification
Error Summary
date_understanding: 7 errors (date calculation)
geometric_shapes: 42 errors (22 circle K→A, 20 rectangle K→H)
Verification
All corrections verified by 6+ language models AND mathematical proof (SVG coordinate analysis).
提供机构：
Zenodo创建时间：
2025-12-30
