SPRINGLab/BPCC_cleaned

--- dataset_info: features: - name: src_lang dtype: class_label: names: '0': eng_Latn '1': ben_Beng '2': guj_Gujr '3': hin_Deva '4': kan_Knda '5': mal_Mlym '6': mar_Deva '7': tam_Taml '8': tel_Telu - name: tgt_lang dtype: class_label: names: '0': eng_Latn '1': ben_Beng '2': guj_Gujr '3': hin_Deva '4': kan_Knda '5': mal_Mlym '6': mar_Deva '7': tam_Taml '8': tel_Telu - name: src_text dtype: string - name: tgt_text dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 1698080949 num_examples: 3990964 download_size: 817154884 dataset_size: 1698080949 configs: - config_name: default data_files: - split: train path: data/train-* task_categories: - translation language: - bn - hi - ta - te - mr - ml - kn - gu size_categories: - 1M<n<10M --- A curated subset of [Bharat Parallel Corpus Collection](https://bpcc.ai4bharat.org/) (BPCC) for 8 Indian languages. Translation pairs are filtered with LABSE score(>0.9) and further preprocessed. Useful for training high-quality translation models.

原始信息汇总

数据集概述

数据集特征

• src_lang: 源语言代码，分类标签包括：

  • 0: eng_Latn
  • 1: ben_Beng
  • 2: guj_Gujr
  • 3: hin_Deva
  • 4: kan_Knda
  • 5: mal_Mlym
  • 6: mar_Deva
  • 7: tam_Taml
  • 8: tel_Telu

• tgt_lang: 目标语言代码，分类标签与src_lang相同。

• src_text: 源文本，数据类型为字符串。

• tgt_text: 目标文本，数据类型为字符串。

• score: 评分，数据类型为float64。

数据集分割

• train: 训练集
  • 数据量: 1698080949 字节
  • 示例数量: 3990964

数据集大小

• 下载大小: 817154884 字节
• 数据集大小: 1698080949 字节