irds/mmarco_v2_de_train

`mmarco/v2/de/train`数据集由ir-datasets包提供，主要用于文本检索任务。该数据集包含查询（queries）、相关性评估（qrels）和文档对（docpairs）三个部分。查询数量为808,731，相关性评估数量为532,761，文档对数量为39,780,811。文档部分需要使用`irds/mmarco_v2_de`数据集。

The `mmarco/v2/de/train` dataset, provided by the ir-datasets package, is primarily intended for text retrieval tasks. This dataset comprises three components: queries, relevance judgments (qrels), and docpairs. There are 808,731 queries, 532,761 relevance judgments, and 39,780,811 docpairs. The document corpus requires the use of the `irds/mmarco_v2_de` dataset.

原始信息汇总

数据集概述

数据集名称

mmarco/v2/de/train

数据来源

• 源数据集：irds/mmarco_v2_de

任务类别

• 文本检索

数据内容

• queries（查询）：数量为808,731
• qrels（相关性评估）：数量为532,761
• docpairs（文档对）：数量为39,780,811

数据使用

• 使用irds/mmarco_v2_de数据集中的docs

数据集加载示例

python from datasets import load_dataset

加载查询数据

queries = load_dataset(irds/mmarco_v2_de_train, queries) for record in queries: record # {query_id: ..., text: ...}

加载相关性评估数据

qrels = load_dataset(irds/mmarco_v2_de_train, qrels) for record in qrels: record # {query_id: ..., doc_id: ..., relevance: ..., iteration: ...}

加载文档对数据

docpairs = load_dataset(irds/mmarco_v2_de_train, docpairs) for record in docpairs: record # {query_id: ..., doc_id_a: ..., doc_id_b: ...}

引用信息

@article{Bonifacio2021MMarco, title={{mMARCO}: A Multilingual Version of {MS MARCO} Passage Ranking Dataset}, author={Luiz Henrique Bonifacio and Israel Campiotti and Roberto Lotufo and Rodrigo Nogueira}, year={2021}, journal={arXiv:2108.13897} }