thoshiths/miracl-multilingual-4M

MIRACL多语言维基百科语料库是一个包含400万段落的精选多语言数据集，源自MIRACL数据集，用于信息检索研究和任务。该语料库涵盖英语、西班牙语、法语和德语四种语言，每种语言包含100万段落，通过水库采样从维基百科转储中获取，以确保代表性覆盖。数据集结构包括文档ID、标题、文本、语言代码、URL和原始MIRACL文档ID。数据基于CC BY-SA 4.0许可证发布。

The MIRACL Multilingual Wikipedia Corpus is a curated 4-million-passage multilingual dataset derived from the MIRACL dataset, sampled for use in Information Retrieval research and assignments. It covers four languages: English, Spanish, French, and German, with 1 million passages per language, obtained via reservoir sampling from Wikipedia dumps to ensure representative coverage. The dataset structure includes document ID, title, text, language code, URL, and original MIRACL document ID. The data is licensed under CC BY-SA 4.0.