BEIR (Benchmarking IR)

BEIR（Benchmarking IR）是包含不同信息检索（IR）任务的异构基准。通过 BEIR，可以系统地研究多种神经检索方法的零样本泛化能力。该基准包含来自 17 个不同数据集的总共 9 个信息检索任务（事实检查、引文预测、重复问题检索、参数检索、新闻检索、问答、推文检索、生物医学 IR、实体检索）：MS MARCO TREC-COVID NFCorpus BioASQ 自然问题 HotpotQA FiQA-2018 Signal-1M TREC-新闻 ArguAna Touche 2020 CQADupStack Quora 问题对 DBPedia SciDocs FEVER Climate-FEVER SciFact

BEIR (Benchmarking IR) is a heterogeneous benchmark encompassing diverse information retrieval (IR) tasks. It enables systematic investigation of the zero-shot generalization capabilities of various neural retrieval methods. This benchmark comprises a total of 9 IR tasks drawn from 17 distinct datasets, including: fact checking, citation prediction, duplicate question retrieval, parameter retrieval, news retrieval, question answering, tweet retrieval, biomedical IR, and entity retrieval. The 17 datasets are as follows: MS MARCO, TREC-COVID, NFCorpus, BioASQ, Natural Questions, HotpotQA, FiQA-2018, Signal-1M, TREC-News, ArguAna, Touche 2020, CQADupStack, Quora Question Pairs, DBPedia, SciDocs, FEVER, Climate-FEVER, SciFact.

搜集汇总

背景与挑战

背景概述

BEIR (Benchmarking IR) 是一个异构信息检索基准，由达姆施塔特工业大学于2021年发布，采用Apache 2.0许可证。它整合了17个数据集，涵盖9个检索任务，旨在系统评估神经检索模型的零样本泛化性能。