irds/msmarco-document-v2

`msmarco-document-v2`数据集由`ir-datasets`包提供，包含11,959,635个文档，主要用于文本检索任务。该数据集被多个相关数据集使用，如`msmarco-document-v2_trec-dl-2019`和`msmarco-document-v2_trec-dl-2020`等。数据集的使用示例展示了如何通过`datasets`库加载数据。

The `msmarco-document-v2` dataset, provided by the `ir-datasets` package, contains 11,959,635 documents and is primarily intended for text retrieval tasks. This dataset is leveraged by multiple associated downstream datasets, including but not limited to `msmarco-document-v2_trec-dl-2019` and `msmarco-document-v2_trec-dl-2020`. Usage examples for the dataset demonstrate how to load data via the `datasets` library.

原始信息汇总

数据集概述

数据集名称

msmarco-document-v2

数据来源

由ir-datasets提供。

数据内容

• 数据类型: 文档
• 数量: 11,959,635 篇文档

使用场景

该数据集被用于以下场景：

• msmarco-document-v2_trec-dl-2019
• msmarco-document-v2_trec-dl-2019_judged
• msmarco-document-v2_trec-dl-2020
• msmarco-document-v2_trec-dl-2020_judged

数据加载示例

python from datasets import load_dataset

docs = load_dataset(irds/msmarco-document-v2, docs) for record in docs: record # {doc_id: ..., url: ..., title: ..., headings: ..., body: ...}

引用信息

@inproceedings{Bajaj2016Msmarco, title={MS MARCO: A Human Generated MAchine Reading COmprehension Dataset}, author={Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, Tong Wang}, booktitle={InCoCo@NIPS}, year={2016} }