google/FACTS-grounding-public

FACTS Grounding 1.0 Public Examples数据集由Google DeepMind和Google Research开发，旨在评估大型语言模型（LLMs）在事实性和基于上下文回答问题的能力。数据集包含860个公开示例，每个示例包括系统提示（`system_instruction`）、用户请求（`user_request`）和长文档（`context_document`）。系统提示提供了一般指令，要求模型仅根据给定上下文回答问题；用户请求包含具体问题；长文档包含回答问题所需的信息。此外，数据集还包含评估提示（`evaluation_prompts.csv`），用于评判模型生成的回答。

The FACTS Grounding dataset, developed by Google DeepMind and Google Research, is designed to evaluate the performance of AI models in terms of factuality and grounding. The dataset contains 860 human-crafted examples for evaluating how well an AI system grounds its answers based on a given context. Each example includes a system prompt, a task, and a long document. Additionally, the dataset contains evaluation prompts for judging model-generated responses to the examples. While this benchmark represents progress in evaluating factual accuracy, there are still limitations, such as relying on potentially noisy automated LLM judge models and focusing only on evaluating responses grounded in long-form text input.