MaLA-LM/mala-opus-dedup-2410
收藏Hugging Face2025-11-17 更新2025-07-05 收录
下载链接：
https://hf-mirror.com/datasets/MaLA-LM/mala-opus-dedup-2410
下载链接
链接失效反馈官方服务：
资源简介：
MaLA语料库（大规模语言适应）是一系列综合性的多语言数据集，旨在支持大型语言模型的持续预训练。mala-opus-dedup-2410是MaLA语料库的双语部分，包含29,202种语言对的平行语料，是经过清理和去重的OPUS语料库版本，可用于多语言翻译模型的训练和微调。
The MaLA Corpus (Massive Language Adaptation) is a series of comprehensive, multilingual datasets designed to support the continual pre-training of large language models. The mala-opus-dedup-2410 is the bilingual part of the MaLA Corpus, containing parallel data in 29,202 language pairs, which is a cleaned and deduplicated version of the OPUS corpus and can be used for training and fine-tuning multilingual translation models.
提供机构：
MaLA-LM
