wecover/OPUS_News-Commentary

--- configs: - config_name: default data_files: - split: train path: '*/*/train.parquet' - split: valid path: '*/*/valid.parquet' - split: test path: '*/*/test.parquet' - config_name: ar data_files: - split: train path: '*/*ar*/train.parquet' - split: test path: '*/*ar*/test.parquet' - split: valid path: '*/*ar*/valid.parquet' - config_name: cs data_files: - split: train path: '*/*cs*/train.parquet' - split: test path: '*/*cs*/test.parquet' - split: valid path: '*/*cs*/valid.parquet' - config_name: de data_files: - split: train path: '*/*de*/train.parquet' - split: test path: '*/*de*/test.parquet' - split: valid path: '*/*de*/valid.parquet' - config_name: en data_files: - split: train path: '*/*en*/train.parquet' - split: test path: '*/*en*/test.parquet' - split: valid path: '*/*en*/valid.parquet' - config_name: es data_files: - split: train path: '*/*es*/train.parquet' - split: test path: '*/*es*/test.parquet' - split: valid path: '*/*es*/valid.parquet' - config_name: fr data_files: - split: train path: '*/*fr*/train.parquet' - split: test path: '*/*fr*/test.parquet' - split: valid path: '*/*fr*/valid.parquet' - config_name: it data_files: - split: train path: '*/*it*/train.parquet' - split: test path: '*/*it*/test.parquet' - split: valid path: '*/*it*/valid.parquet' - config_name: ja data_files: - split: train path: '*/*ja*/train.parquet' - split: test path: '*/*ja*/test.parquet' - split: valid path: '*/*ja*/valid.parquet' - config_name: nl data_files: - split: train path: '*/*nl*/train.parquet' - split: test path: '*/*nl*/test.parquet' - split: valid path: '*/*nl*/valid.parquet' - config_name: pt data_files: - split: train path: '*/*pt*/train.parquet' - split: test path: '*/*pt*/test.parquet' - split: valid path: '*/*pt*/valid.parquet' - config_name: ru data_files: - split: train path: '*/*ru*/train.parquet' - split: test path: '*/*ru*/test.parquet' - split: valid path: '*/*ru*/valid.parquet' - config_name: hi data_files: - split: train path: '*/*hi*/train.parquet' - split: test path: '*/*hi*/test.parquet' - split: valid path: '*/*hi*/valid.parquet' - config_name: id data_files: - split: train path: '*/*id*/train.parquet' - split: test path: '*/*id*/test.parquet' - split: valid path: '*/*id*/valid.parquet' - config_name: kk data_files: - split: train path: '*/*kk*/train.parquet' - split: test path: '*/*kk*/test.parquet' - split: valid path: '*/*kk*/valid.parquet' ---

原始信息汇总

数据集配置

默认配置

• 训练集: */*/train.parquet
• 验证集: */*/valid.parquet
• 测试集: */*/test.parquet

阿拉伯语配置

• 训练集: */*ar*/train.parquet
• 验证集: */*ar*/valid.parquet
• 测试集: */*ar*/test.parquet

捷克语配置

• 训练集: */*cs*/train.parquet
• 验证集: */*cs*/valid.parquet
• 测试集: */*cs*/test.parquet

德语配置

• 训练集: */*de*/train.parquet
• 验证集: */*de*/valid.parquet
• 测试集: */*de*/test.parquet

英语配置

• 训练集: */*en*/train.parquet
• 验证集: */*en*/valid.parquet
• 测试集: */*en*/test.parquet

西班牙语配置

• 训练集: */*es*/train.parquet
• 验证集: */*es*/valid.parquet
• 测试集: */*es*/test.parquet

法语配置

• 训练集: */*fr*/train.parquet
• 验证集: */*fr*/valid.parquet
• 测试集: */*fr*/test.parquet

意大利语配置

• 训练集: */*it*/train.parquet
• 验证集: */*it*/valid.parquet
• 测试集: */*it*/test.parquet

日语配置

• 训练集: */*ja*/train.parquet
• 验证集: */*ja*/valid.parquet
• 测试集: */*ja*/test.parquet

荷兰语配置

• 训练集: */*nl*/train.parquet
• 验证集: */*nl*/valid.parquet
• 测试集: */*nl*/test.parquet

葡萄牙语配置

• 训练集: */*pt*/train.parquet
• 验证集: */*pt*/valid.parquet
• 测试集: */*pt*/test.parquet

俄语配置

• 训练集: */*ru*/train.parquet
• 验证集: */*ru*/valid.parquet
• 测试集: */*ru*/test.parquet

印地语配置

• 训练集: */*hi*/train.parquet
• 验证集: */*hi*/valid.parquet
• 测试集: */*hi*/test.parquet

印尼语配置

• 训练集: */*id*/train.parquet
• 验证集: */*id*/valid.parquet
• 测试集: */*id*/test.parquet

哈萨克语配置

• 训练集: */*kk*/train.parquet
• 验证集: */*kk*/valid.parquet
• 测试集: */*kk*/test.parquet