Experimental-Orange/gsm-noop-audited

GSM-NoOp（audited）是一个用于数学推理和大型语言模型评估的数据集，基于Apple的GSM-Symbolic问题集生成。它包含为GSM-Symbolic问题添加的无关干扰子句（NoOp distractors），这些子句经过独立模型（如GPT-5.5）的审核，以确保其真正无关性（即不干扰问题解答）。数据集旨在重新评估前沿模型（如Opus 4.6、Haiku 4.5、GPT-4o）在遇到无关信息时的表现，并区分真正无关的子句与模糊子句。内容涵盖原始问题、添加干扰子句的问题、审核结果（包括真正无关、模糊或实际相关的分类）以及多个模型在不同数据集（基础集、未审核集、审核集）上的评估结果。数据集用于研究模型在数学推理任务中的鲁棒性和准确性。GSM-NoOp (audited) is a dataset for mathematical reasoning and large language model evaluation, generated based on Apple's GSM-Symbolic problem set. It incorporates irrelevant distractor clauses (NoOp distractors) added to the original GSM-Symbolic problems, and these distractor clauses have been audited by independent models such as GPT-5.5 to verify their genuine irrelevance—meaning they do not interfere with problem-solving. The dataset is designed to re-evaluate the performance of state-of-the-art models (e.g., Opus 4.6, Haiku 4.5, GPT-4o) when confronted with irrelevant information, and to differentiate between truly irrelevant clauses and ambiguous ones. It includes original problems, problems with added distractor clauses, audit results (comprising classifications of clauses as genuinely irrelevant, ambiguous, or actually relevant), as well as evaluation results of multiple models across three distinct dataset splits: the base set, unaudited set, and audited set. This dataset is employed to research the robustness and accuracy of models in mathematical reasoning tasks.