GAIR/FRoG

FRoG是一个用于评估模型在模糊推理和广义量词处理方面能力的基准数据集。数据集中的问题来源于现实世界的数学应用题，并引入了模糊性。数据集包含多个配置（mask_percent, mask_quant, mislead, X），每个配置下分为easy和hard两个子集。每个问题包含多个特征，如问题ID、问题文本、目标百分比提及、量词、量词强度差距、答案、原始问题、原始推理链、原始问题文本和来源。

The FRoG dataset is a fuzzy reasoning benchmark designed to evaluate the fuzzy reasoning abilities of models, particularly in handling generalized quantifiers. The dataset includes multiple configurations, each with easy and hard difficulty levels. The features of the dataset include question ID, question text, target percentage mention, quantifier, quantifier strength gap, answer, original question, original reasoning, raw question (excluding choices), and source. The dataset is used for multiple-choice tasks, with the language being English and tags including Natural Language Processing, Generalized Quantifiers, Fuzzy Reasoning, and Mathematical Reasoning.