qintongli/GSM-Plus

GSM-Plus数据集旨在通过测试广泛的问题变体来评估大型语言模型（LLMs）在数学推理能力上的鲁棒性。该数据集是基于GSM8K数据集扩展的对抗性小学数学数据集，包含了五种数学扰动类型：数值变化、算术变化、问题理解、干扰项插入和批判性思维。每种扰动类型下又细分为若干子类别。基于GSM8K的1,319个测试问题，每个问题生成了八个变体，最终形成了包含10,552个问题变体的GSM-PLUS数据集。

GSM-Plus aims to evaluate the robustness of LLMs math reasoning capability by testing a wide range of question variations. GSM-Plus is an adversarial grade school math dataset, an extension of GSM8K augmented with various mathematical perturbations. Motivated by the capability taxonomy for solving math problems mentioned in Polya’s principles, we identify 5 perspectives to guide the development of GSM-PLUS: numerical variation, arithmetic variation, problem understanding, distractor insertion, and critical thinking. Based on the 1,319 test questions from GSM8K, we create eight variations for each question, yielding GSM-PLUS comprising 10,552 question variations.

原始信息汇总

数据集许可证

• 许可证类型: CC BY-SA 4.0

搜集汇总

背景与挑战

背景概述

GSM-Plus is a benchmark dataset for evaluating LLMs' robustness in solving math problems, featuring 10,552 adversarial variations of grade school math questions. It tests numerical and arithmetic variations, problem understanding, distractor insertion, and critical thinking, derived from the GSM8K dataset. The dataset is licensed under CC BY-SA 4.0 and is designed for testing purposes only.