irds/wikiclir_ja

`wikiclir/ja`数据集由`ir-datasets`包提供，包含文档、查询和相关性评估三个部分。文档部分包含1,071,292条记录，查询部分包含426,431条记录，相关性评估部分包含3,338,667条记录。该数据集用于文本检索任务。

原始信息汇总

数据集卡片 wikiclir/ja

数据集概述

wikiclir/ja 数据集由 ir-datasets 包提供。

数据内容

该数据集包含以下部分：

• docs：文档（即语料库），数量为 1,071,292。
• queries：查询（即主题），数量为 426,431。
• qrels：相关性评估，数量为 3,338,667。

使用方法

以下是加载和使用该数据集的示例代码： python from datasets import load_dataset

docs = load_dataset(irds/wikiclir_ja, docs) for record in docs: record # {doc_id: ..., title: ..., text: ...}

queries = load_dataset(irds/wikiclir_ja, queries) for record in queries: record # {query_id: ..., text: ...}

qrels = load_dataset(irds/wikiclir_ja, qrels) for record in qrels: record # {query_id: ..., doc_id: ..., relevance: ..., iteration: ...}

引用信息

@inproceedings{sasaki-etal-2018-cross, title = "Cross-Lingual Learning-to-Rank with Shared Representations", author = "Sasaki, Shota and Sun, Shuo and Schamoni, Shigehiko and Duh, Kevin and Inui, Kentaro", booktitle = "Proceedings of the 2018 Conference of the North {A}merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)", month = jun, year = "2018", address = "New Orleans, Louisiana", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/N18-2073", doi = "10.18653/v1/N18-2073", pages = "458--463" }