MURU-BENCH: A Benchmark for Mathematical Reasoning Under Uncertainty
收藏DataCite Commons2026-05-05 更新2026-05-07 收录
下载链接：
https://zenodo.org/doi/10.5281/zenodo.20036750
下载链接
链接失效反馈官方服务：
资源简介：
MURU-BENCH is a 3,000-problem benchmark for evaluating mathematical reasoning under uncertainty in large language models. Unlike existing benchmarks (GSM8K, MATH, MMLU) that score correctness as a single number, MURU-BENCH requires every prediction to include a point estimate, a calibrated confidence interval, a stated confidence probability, and an identified reasoning framework.
Problems span five categories:- Bayesian Updating (910 problems)- Distribution Estimation (660 problems)- Decision Under Uncertainty (525 problems)- Adversarial Ambiguity (474 problems)- Conditional Probability Chains (431 problems)
Five difficulty levels (D1-D5) are generated by 21 parametric templates with closed-form solutions, guaranteeing correctness by construction. The evaluation harness reports six metrics including Expected Calibration Error and a safety-relevant overconfidence rate, and supports six API providers (OpenAI, Anthropic, Google, Groq, OpenRouter, Together AI).
GitHub: https://github.com/swetank18/MURU_proj
提供机构：
Zenodo
创建时间：
2026-05-05
