launch/FactRBench

FactRBench是一个用于评估大型语言模型生成的长篇回答真实性的基准数据集，关注精确性和召回率。它包括factbench和reddit两个部分，factbench部分包含原始FactBench数据集的提示，并注释有从顶级LLM模型回答中提取的参考事实；reddit部分包含来自多个子版块的提示和高质量的人类回答，形成具有高多样性的参考事实集。FactRBench is a benchmark dataset for evaluating the factuality of long-form responses generated by large language models (LLMs), focusing on both precision and recall. It includes two parts: factbench and reddit. The factbench part contains prompts from the original FactBench dataset, annotated with reference facts extracted from responses by top LLMs; the reddit part includes prompts from various subreddits paired with high-quality human answers to form diverse reference fact sets.

搜集汇总

背景与挑战

背景概述

FactRBench是一个用于评估大型语言模型生成长文本响应事实性的基准数据集，专注于精确性和召回率。它包含两个子集：factbench（基于原始FactBench提示，并标注了从GPT-4o、Claude等顶级模型响应中提取的参考事实）和reddit（基于Reddit社区的人类答案，提供高多样性参考事实）。每个数据点包括问题字符串和从模型或用户答案中提取的事实集合字典，旨在支持全面的事实性评估。