ByteDance-Seed/BeyondAIME

BeyondAIME是一个专门设计用来评估高级数学推理能力的测试集。问题来源于高中和大学数学竞赛，难度不低于AIME的第11-15题。每个问题都是经过手动修订的唯一问题，以确保不会出现在标准预训练语料库中，从而真正测试模型的推理能力。数据集专注于推理而非知识，并确保问题设计合理，避免伪证明问题。每个问题的答案都是正整数，方便进行自动准确评估。BeyondAIME is a curated test set designed to benchmark advanced mathematical reasoning. The problems are sourced from high-school and university mathematics competitions, with a difficulty level greater than or equal to that of AIME Problems #11-15. Each problem has been manually revised to be unique, ensuring it will not be found in standard pre-training corpora and providing a true test of a models reasoning abilities. The dataset focuses on reasoning rather than knowledge and ensures robust problem design to avoid pseudo-proof problems. Each problems answer is a positive integer, allowing for unambiguous and 100% accurate automated verification of model performance.

搜集汇总

背景与挑战

背景概述

BeyondAIME是一个专注于高级数学推理评估的数据集，包含100个难度高于或等于AIME竞赛高难度题目的数学问题，每个问题都有唯一的整数答案，旨在测试模型的推理能力而非知识记忆。数据集经过人工修订以确保独特性，避免预训练数据污染，并支持自动化评估。