Retrieval-Augmented Generation Benchmark (RGB)

Retrieval-Augmented Generation Benchmark (RGB)是由中国科学院软件研究所创建的一个新型数据集，旨在评估大型语言模型在检索增强生成方面的能力。该数据集包含600个基础问题，涵盖英语和中文，通过最新新闻文章构建，并使用搜索引擎获取外部文档。RGB数据集特别关注四大基本能力：噪声鲁棒性、否定拒绝、信息整合和反事实鲁棒性，旨在解决大型语言模型在处理外部信息时的挑战，如信息噪声和事实错误。

Retrieval-Augmented Generation Benchmark (RGB) is a novel dataset developed by the Institute of Software, Chinese Academy of Sciences, designed to evaluate the capabilities of large language models (LLMs) in retrieval-augmented generation. This dataset includes 600 foundational questions covering both English and Chinese, constructed from recent news articles, with external documents retrieved via search engines. The RGB benchmark specifically focuses on four core capabilities: noise robustness, negation rejection, information integration, and counterfactual robustness. It aims to address the challenges faced by large language models when processing external information, such as informational noise and factual errors.

搜集汇总

背景与挑战

背景概述

Retrieval-Augmented Generation Benchmark (RGB)是由中国科学院软件研究所创建的数据集，用于评估大型语言模型在检索增强生成方面的能力。它包含600个英语和中文问题，基于新闻文章构建，并关注噪声鲁棒性、否定拒绝、信息整合和反事实鲁棒性四大能力，以解决模型处理外部信息时的挑战。