MLDoc (Multilingual Document Classification Corpus)

跨语言文档分类旨在用一种语言的资源训练文档分类器，并将其转换为另一种语言，而无需任何额外资源。文献中提出了几种方法，目前的最佳实践是在路透社语料库第 2 卷的子集上对其进行评估。但是，该子集仅涵盖少数语言（英语、德语、法语和西班牙语），并且几乎所有已发表的作品都集中在关于英语和德语之间的转换。此外，我们观察到语言之间的类先验分布显着不同。我们认为这使多语言的评估变得复杂。

Cross-lingual document classification aims to train a document classifier using resources from one language and adapt it to another language without any additional resources. Several methods have been proposed in the literature, and current best practices evaluate these methods on a subset of the Reuters Corpus Volume 2. However, this subset only covers a small number of languages (English, German, French, and Spanish), and nearly all published works focus on cross-lingual transfer between English and German. Furthermore, we observed that the class prior distributions differ significantly across languages, which we argue complicates multilingual evaluation.

搜集汇总

背景与挑战

背景概述

MLDoc是一个用于跨语言文档分类的数据集，旨在通过单一语言资源训练分类器并迁移到其他语言。现有评估主要基于路透社语料库的有限语言子集（如英语和德语），且数据中语言间的类分布差异可能增加多语言评估的复杂性。