Complex Attributed Question Answering (CAQA)
收藏arXiv2024-01-26 更新2024-08-06 收录
下载链接：
http://arxiv.org/abs/2401.14640v1
下载链接
链接失效反馈官方服务：
资源简介：
本研究引入了Complex Attributed Question Answering (CAQA)数据集，由东南大学计算机科学与工程学院开发，旨在通过知识图谱(KGs)评估大型语言模型(LLMs)在复杂问答归属任务中的性能。CAQA数据集包含161,174条记录，通过自动方法生成，涵盖支持性、不足、矛盾和无关四种细粒度归属类别，以及单、联合、交集和串联四种归属复杂度类型。该数据集用于开发和选择LLM归属评估器，特别是在识别归属错误和复杂引用-声明推理方面。
This study introduces the Complex Attributed Question Answering (CAQA) dataset, developed by the School of Computer Science and Engineering, Southeast University. This dataset is designed to evaluate the performance of Large Language Models (LLMs) on complex question answering attribution tasks using Knowledge Graphs (KGs). Comprising 161,174 automatically generated records, CAQA covers four fine-grained attribution categories: supportive, insufficient, contradictory, and irrelevant, as well as four attribution complexity types: single, joint, intersection, and sequential. This dataset is utilized for developing and selecting LLM attribution evaluators, particularly for detecting attribution errors and complex reference-claim reasoning.
提供机构：
东南大学计算机科学与工程学院
创建时间：
2024-01-26
