XATU

XATU是由加州大学戴维斯分校IFM实验室和Megagon Labs合作开发的一个细粒度指令基准数据集，专注于可解释的文本更新。该数据集包含1000个实例，覆盖了从词汇、句法、语义到知识密集型的多种编辑任务，如简化、语法检查和事实检查等。数据集通过结合大型语言模型（LLM）和人工标注，确保了指令的细粒度和编辑解释的高标准。XATU旨在通过提供详细的指令和解释，评估和提升大型语言模型在文本编辑任务中的性能，特别是在理解用户意图和提供透明编辑决策方面。XATU is a fine-grained instruction benchmark dataset co-developed by the IFM Lab at the University of California, Davis and Megagon Labs, focusing on explainable text updates. This dataset contains 1,000 instances, covering a wide range of editing tasks spanning from lexical, syntactic, semantic to knowledge-intensive scenarios, including simplification, grammar checking, fact checking, and others. The dataset ensures high standards for both instruction granularity and editing explanations by integrating large language models (LLMs) and human annotations. XATU aims to evaluate and enhance the performance of large language models on text editing tasks, particularly in terms of understanding user intent and providing transparent editing decisions, by offering detailed instructions and corresponding explanations.

搜集汇总

背景与挑战

背景概述

XATU是一个由加州大学戴维斯分校IFM实验室与Megagon Labs合作开发的细粒度指令基准数据集，专注于可解释的文本更新，包含1000个实例，覆盖词汇、句法、语义和知识密集型编辑任务。它通过结合大型语言模型和人工标注确保高质量，旨在评估和提升大型语言模型在文本编辑任务中的性能，特别是在理解用户意图和提供透明决策方面。