FormulaQA

FormulaQA是一个专为公式基础的数值推理设计的数据集，由南京大学软件新技术国家重点实验室创建。该数据集包含5420个问题，所有问题均来自中国的初中物理考试，并要求使用特定的公式进行解答。数据集的创建过程中，结合了人工标注和大型语言模型的辅助，确保每个问题都附有详细的解释文本、最终答案及相关公式。FormulaQA的应用领域主要在于评估和提升模型在复杂、公式驱动的数值推理任务上的表现，特别是在物理教育和技术评估领域。FormulaQA is a dataset specifically designed for formula-based numerical reasoning, created by the State Key Laboratory for Novel Software Technology at Nanjing University. This dataset comprises 5,420 questions, all sourced from Chinese junior high school physics examinations, where each question necessitates the use of specific formulas to arrive at a solution. During the development of the dataset, a combination of manual annotation and assistance from large language models was employed to ensure that every question is paired with detailed explanatory text, the final correct answer, and the relevant formulas used for solving it. The primary application areas of FormulaQA are the evaluation and improvement of models' performance on complex, formula-driven numerical reasoning tasks, especially within the domains of physics education and technical assessment.

搜集汇总

背景与挑战

背景概述

FormulaQA（在HTML中对应FormulaReasoning）是一个用于基于公式数值推理的数据集，包含FormulaReasoning和FormulaReasoning+两个子集，每个子集有训练集和两种测试集（HoF和HeF）。该数据集旨在评估语言模型在公式理解和数值计算方面的推理能力，提供了多种LLM基线和小模型微调方法。