corag/kilt-corpus

KILT语料库包含大约3600万条来自 [Multi-task retrieval for knowledge-intensive tasks](https://arxiv.org/pdf/2101.00117) 论文的Wikipedia段落。它也是该论文中使用的检索语料库 [Chain-of-Retrieval Augmented Generation](https://arxiv.org/pdf/2501.14342)。数据集的字段包括：唯一标识符(id)、Wikipedia页面标题(title)、段落文本内容(contents)以及用于KILT评估的Wikipedia页面唯一标识符(wikipedia_id)。

The KILT Corpus contains approximately 36 million Wikipedia passages from the [Multi-task retrieval for knowledge-intensive tasks](https://arxiv.org/pdf/2101.00117) paper. It is also the retrieval corpus used in the paper [Chain-of-Retrieval Augmented Generation](https://arxiv.org/pdf/2501.14342). The fields of the dataset include a unique identifier (id), the title of the Wikipedia page from which the passage originates (title), the textual content of the passage (contents), and the unique identifier for the Wikipedia page used for KILT evaluation (wikipedia_id).