apple/GSM-Symbolic

GSM-Symbolic数据集用于研究大型语言模型在数学推理方面的局限性。该数据集包含三个变体：GSM-Symbolic、GSM-Symbolic-P1和GSM-Symbolic-P2，难度依次递增。每个变体包含生成的数据样本，用于评估模型性能。数据格式包括问题、答案、原始问题ID、原始问题、原始答案以及用于检测数据泄露的canary字符串。实验设置部分详细描述了提示模板、解码策略和答案提取方法。

The GSM-Symbolic dataset and its variants GSM-Symbolic-P1 and GSM-Symbolic-P2 are used to evaluate the capabilities of large language models in mathematical reasoning, specifically to understand their limitations. The dataset includes multiple fields such as questions, answers, original questions, and original answers for model evaluation. The difficulty of the datasets increases in the order of GSM-Symbolic, GSM-Symbolic-P1, and GSM-Symbolic-P2, with GSM-Symbolic-P2 being the most difficult.