irds/mr-tydi_ko

`mr-tydi/ko`数据集由ir-datasets包提供，包含1,496,126个文档（docs）、2,019个查询（queries）和2,116个相关性评估（qrels）。该数据集用于文本检索任务，并且被多个子数据集（如`mr-tydi_ko_dev`、`mr-tydi_ko_test`和`mr-tydi_ko_train`）使用。用户可以通过Hugging Face的datasets库加载和使用该数据集。

The `mr-tydi/ko` dataset is provided by the ir-datasets package, which contains 1,496,126 documents, 2,019 queries, and 2,116 relevance judgments (qrels). This dataset is designed for text retrieval tasks, and is utilized by multiple sub-datasets such as `mr-tydi_ko_dev`, `mr-tydi_ko_test`, and `mr-tydi_ko_train`. Users can load and use this dataset via Hugging Face's datasets library.

原始信息汇总

数据集概述

数据集名称

mr-tydi/ko

数据来源

由 ir-datasets 提供。

数据内容

• 文档 (docs): 总数为1,496,126篇。
• 查询 (queries): 总数为2,019条。
• 相关性评估 (qrels): 总数为2,116条。

数据用途

该数据集被用于以下数据集：

• mr-tydi_ko_dev
• mr-tydi_ko_test
• mr-tydi_ko_train

数据加载示例

python from datasets import load_dataset

docs = load_dataset(irds/mr-tydi_ko, docs) for record in docs: record # {doc_id: ..., text: ...}

queries = load_dataset(irds/mr-tydi_ko, queries) for record in queries: record # {query_id: ..., text: ...}

qrels = load_dataset(irds/mr-tydi_ko, qrels) for record in qrels: record # {query_id: ..., doc_id: ..., relevance: ..., iteration: ...}

引用信息

@article{Zhang2021MrTyDi, title={{Mr. TyDi}: A Multi-lingual Benchmark for Dense Retrieval}, author={Xinyu Zhang and Xueguang Ma and Peng Shi and Jimmy Lin}, year={2021}, journal={arXiv:2108.08787}, } @article{Clark2020TyDiQa, title={{TyDi QA}: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages}, author={Jonathan H. Clark and Eunsol Choi and Michael Collins and Dan Garrette and Tom Kwiatkowski and Vitaly Nikolaev and Jennimaria Palomaki}, year={2020}, journal={Transactions of the Association for Computational Linguistics} }