"FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs"

"This dataset is constructed to evaluate the reasoning ability of language models on mathematical word problems under fuzzy and generalized quantifier settings. The collection contains over 1,600 items systematically adapted from classic math word problem benchmarks, where explicit numerical expressions are replaced with linguistic quantifier phrases such as \u201cmost\u201d, \u201cabout half\u201d, or \u201ca tiny amount\u201d. Each problem is rewritten into multiple fuzzy variants, including masked percentages, quantifier substitutions, and misleading quantifier options. The dataset covers both easy and hard subsets, and is distributed in standard formats for flexible use. This resource enables the fine-grained evaluation of models in both categorical quantifier prediction and graded fuzzy membership estimation, providing a testbed for research in fuzzy reasoning and natural language understanding."