DaydreamerMZM/SenseMath

--- license: mit task_categories: - question-answering language: - en tags: - math - number-sense - benchmark - shortcuts - numerical-reasoning size_categories: - 1K<n<10K --- # SenseMath: Evaluating Number Sense in Large Language Models **SenseMath** is a controlled benchmark for measuring whether LLMs can exploit number-sense shortcuts. ## Dataset Description - **1,600 item families** across 8 categories and 4 digit scales - **3 variants per family**: strong-shortcut, weak-shortcut, control - **4,800 total items** - **Categories**: Magnitude Estimation, Structural Shortcuts, Relative Distance, Cancellation, Compatible Numbers, Landmark Comparison, Equation Reasoning, Option Elimination - **Digit scales**: d=2, 4, 8, 16 ## Files | File | Description | |------|-------------| | `data/sensemath_v2_d2.json` | 400 families, 2-digit operands | | `data/sensemath_v2_d4.json` | 400 families, 4-digit operands | | `data/sensemath_v2_d8.json` | 400 families, 8-digit operands | | `data/sensemath_v2_d16.json` | 400 families, 16-digit operands | | `data/judge_j1.json` | J1 task: shortcut recognition (251 items) | | `data/judge_j2.json` | J2 task: strategy identification (80 items) | | `data/judge_j3.json` | J3 task items | ## Usage ```python from datasets import load_dataset ds = load_dataset("DaydreamerMZM/SenseMath", split="train") # Or load directly import json with open("data/sensemath_v2_d4.json") as f: families = json.load(f) ``` ## Citation ```bibtex @article{zhuang2025sensemath, title={SenseMath: Evaluating Number Sense in Large Language Models}, author={Zhuang, Haomin and Wang, Xiangqi and Shen, Yili and Cheng, Ying and Zhang, Xiangliang}, journal={arXiv preprint arXiv:XXXX.XXXXX}, year={2025} } ``` ## Links - **Paper**: [arXiv](https://arxiv.org/abs/XXXX.XXXXX) - **Code**: [GitHub](https://github.com/zhmzm/SenseMath) - **Project Page**: [zhmzm.github.io/SenseMath](https://zhmzm.github.io/SenseMath/)--- 许可证：MIT 任务类别： - 问答 语言： - 英语 标签： - 数学 - 数感 - 基准测试 - 捷径 - 数值推理 规模类别： - 1K<n<10K --- # SenseMath：评估大语言模型（Large Language Model）的数感能力 **SenseMath** 是一款受控基准测试集，用于衡量大语言模型能否利用数感捷径。 ## 数据集说明 - **1600个项目族**，覆盖8个类别与4种数字位数尺度 - 每个项目族包含**3个变体**：强捷径、弱捷径、对照组 - **总计4800个项目** - **类别**：幅度估计、结构捷径、相对距离、抵消运算、兼容数字、地标比较、方程推理、选项排除 - **数字位数尺度**：d=2、4、8、16 ## 文件说明 | 文件名 | 描述 | |------|-------------| | `data/sensemath_v2_d2.json` | 包含400个项目族，操作数为2位数字 | | `data/sensemath_v2_d4.json` | 包含400个项目族，操作数为4位数字 | | `data/sensemath_v2_d8.json` | 包含400个项目族，操作数为8位数字 | | `data/sensemath_v2_d16.json` | 包含400个项目族，操作数为16位数字 | | `data/judge_j1.json` | J1任务：捷径识别（共251个项目） | | `data/judge_j2.json` | J2任务：策略识别（共80个项目） | | `data/judge_j3.json` | J3任务项目集 | ## 使用方法 python from datasets import load_dataset ds = load_dataset("DaydreamerMZM/SenseMath", split="train") # 或直接读取文件 import json with open("data/sensemath_v2_d4.json") as f: families = json.load(f) ## 引用格式 bibtex @article{zhuang2025sensemath, title={SenseMath: Evaluating Number Sense in Large Language Models}, author={Zhuang, Haomin and Wang, Xiangqi and Shen, Yili and Cheng, Ying and Zhang, Xiangliang}, journal={arXiv preprint arXiv:XXXX.XXXXX}, year={2025} } ## 相关链接 - **论文**：[arXiv](https://arxiv.org/abs/XXXX.XXXXX) - **代码**：[GitHub](https://github.com/zhmzm/SenseMath) - **项目主页**：[zhmzm.github.io/SenseMath](https://zhmzm.github.io/SenseMath/)