nirmalendu01/ted_multi

TED multi — TSV mirror是一个多语言平行语料库，基于TED演讲内容构建。它是原始neulab/ted_multi数据集的忠实重新托管版本，采用行对齐的多路并行TSV格式，旨在解决原HuggingFace数据集脚本损坏的问题。数据集包含训练集（约258k行）、开发集（约6k行）和测试集（约7k行），每个TSV行包含60个语言列以及talk_name和id列。缺失的翻译用__NULL__标记，用户可根据需要过滤这些行以获取N路平行子集。该语料库适用于神经机器翻译、多语言自然语言处理等任务，原始来源为Qi等人在NAACL 2018发表的论文《When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?》。数据集语言为多语言，许可证为cc-by-nc-nd-4.0，规模在100K到1M之间。TED multi — TSV mirror is a faithful re-host of the original neulab/ted_multi TED Talks corpus, in the same row-aligned multi-way parallel TSV format. It was created because the HF Datasets script for neulab/ted_multi is broken, and this mirror does not modify the data. The dataset includes train split (≈258k rows), dev split (≈6k rows), and test split (≈7k rows). Each TSV row has 60 language columns plus talk_name and id, with missing translations written as __NULL__. Users can drop rows with __NULL__ to obtain an N-way parallel subset. It is a multilingual parallel corpus based on TED Talks, suitable for tasks like neural machine translation. Source: Qi, Y., et al. (2018). When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation? In NAACL. The dataset is multilingual, licensed under cc-by-nc-nd-4.0, with size category 100K<n<1M.