nirmalendu01/ted_multi

---
language:
- multilingual
license: cc-by-nc-nd-4.0
pretty_name: TED multi (4-way TSV mirror)
tags:
- parallel-corpora
- tedtalks
- multilingual
- re-host
size_categories:
- 100K<n<1M
---

# TED multi — TSV mirror

Faithful re-host of the original [`neulab/ted_multi`](https://github.com/neulab/word-embeddings-for-nmt)
TED Talks corpus, in the same row-aligned multi-way parallel TSV format that
was distributed at `https://www.phontron.com/data/ted_talks.tar.gz`.

The HF Datasets script `neulab/ted_multi` is currently broken (`_DATA_URL`
returns an SPA), which is why this mirror exists. It does not modify the data.

## Files

- `all_talks_train.tsv` — train split (≈258k rows).
- `all_talks_dev.tsv` — dev split (≈6k rows).
- `all_talks_test.tsv` — test split (≈7k rows).

## Schema

Each TSV row has **60 language columns + `talk_name` + `id`** (depending on
header line — see the first line of each file). Missing translations for a row
are written literally as `__NULL__` (some legacy snapshots also use
`_ _ NULL _ _`). When you need an `N`-way parallel subset, drop any row where
any of the target language columns equals `__NULL__`.

## Source / attribution

Qi, Y., Sachan, D., Felix, M., Padmanabhan, S., & Neubig, G. (2018).
*When and Why are Pre-trained Word Embeddings Useful for Neural Machine
Translation?* In NAACL.

Original distribution lived at <https://www.phontron.com/data/ted_talks.tar.gz>.