opendatalab/CiteVQA

CiteVQA是一个文档视觉问答基准，专注于忠实证据归因。与传统DocVQA数据集仅评估最终答案不同，CiteVQA要求模型在元素级别上基于源文档提供证据来回答问题。该基准设计用于评估系统是否不仅能正确回答，还能在长且真实的PDF中引用正确的支持区域。数据集包含1,897个问题，基于711个PDF，涵盖7个宏观领域和30个子领域，平均每个文档40.6页，覆盖英文和中文文档，包括单文档和多文档设置。

CiteVQA is a document visual question answering benchmark for faithful evidence attribution. Unlike conventional DocVQA datasets that only score the final answer, CiteVQA requires a model to answer a question with evidence grounded in the source document at the element level. The benchmark is designed to evaluate whether a system can not only answer correctly, but also cite the right supporting region in long, real-world PDFs. The dataset contains 1,897 questions built from 711 PDFs across 7 macro-domains and 30 sub-domains, with an average of 40.6 pages per document. It covers both English and Chinese documents, and includes single-document as well as multi-document settings.