检索增强生成RagQA

该Benchmark主要将Rag领域的单跳、多跳以及垂域的benchmark合并如TriviaQA、Musique、PubmedQA等，方便用于评测生成模型的rag能力(固定query的doc list), 用于评测Rag排序(提供doc list pool。

This benchmark primarily integrates single-hop, multi-hop, and domain-specific benchmark datasets within the retrieval-augmented generation (RAG) domain, including TriviaQA, Musique, PubmedQA, among others. It serves two main evaluation scenarios: one is to assess the RAG capabilities of generative models, where fixed queries and their corresponding document lists are provided; the other is to evaluate RAG ranking performance, where a document list pool and queries are provided.

搜集汇总

背景与挑战

背景概述

该数据集整合了六个广泛使用的开放域问答数据集，用于评估检索准确性和答案生成性能，支持多跳和单跳推理场景。它包含查询、正确答案、相关文档和干扰文档等字段，并提供了一个基于维基百科的预处理文档池，便于检索模型的高效训练和评估。