FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs

This dataset is constructed to evaluate the reasoning ability of language models on mathematical word problems under fuzzy and generalized quantifier settings. The collection contains over 1,600 items systematically adapted from classic math word problem benchmarks, where explicit numerical expressions are replaced with linguistic quantifier phrases such as \u201cmost\u201d, \u201cabout half\u201d, or \u201ca tiny amount\u201d. Each problem is rewritten into multiple fuzzy variants, including masked percentages, quantifier substitutions, and misleading quantifier options. The dataset covers both easy and hard subsets, and is distributed in standard formats for flexible use. This resource enables the fine-grained evaluation of models in both categorical quantifier prediction and graded fuzzy membership estimation, providing a testbed for research in fuzzy reasoning and natural language understanding.

本数据集旨在评估语言模型在模糊与广义量词场景下的数学应用题推理能力。该数据集包含1600余条题目，均系统改编自经典数学应用题基准数据集，其中原有的显式数值表达式被替换为诸如"most（大多数）"、"about half（约半数）"及"a tiny amount（极少量）"这类语言量化短语。每个题目会被改写为多种模糊变体，涵盖掩码百分比、量词替换与误导性量词选项三类形式。本数据集包含简单与困难两个子集，并以标准格式发布以支持灵活使用。该资源可实现对模型在类别型量词预测与分级模糊隶属度估计（graded fuzzy membership estimation）两类任务中的细粒度评估，为模糊推理与自然语言理解（Natural Language Understanding）领域的研究提供了专用测试平台。