Audio-NTREX-4L

# Audio-NTREX-4L <p align="center"> <img src="Audio_NTREX_4L.png" width="400" alt="logo"> </p> ## Dataset Description **Audio-NTREX-4L** is a long-form multilingual speech translation dataset from 🇫🇷 French, 🇪🇸 Spanish, 🇵🇹 Portuguese and 🇩🇪 German to 🇬🇧 English designed to evaluate speech translation models on multi-sentence utterances. It is built from the text translation dataset [NTREX](https://github.com/MicrosoftTranslator/NTREX) by aggregating multiple sentences from a same context to create new source texts and their reference translation. We then use 3 different state-of-the-art commercial Text-To-Speech systems from [ElevenLabs](https://elevenlabs.io/text-to-speech), [Cartesia](https://cartesia.ai/sonic) and [Gradium](https://gradium.ai/#models) to synthesize the source texts into speech. We condition audio generations using voices from the multilingual [CML-TTS](https://www.openslr.org/146/) dataset. --- ## Dataset Summary * **Original data:** [NTREX](https://github.com/MicrosoftTranslator/NTREX) * **Source modalities:** Audio, Text * **Target modality:** Text * **Source languages:** French, Spanish, Portuguese, German * **Target language:** English * **Total number of source/target pairs:** 3600 * **Number of unique source texts per language:** 300 * **Average source sample duration:** 45 seconds --- ## Dataset Construction We use the following files containing text translation data from the [NTREX-128](https://github.com/MicrosoftTranslator/NTREX/tree/main/NTREX-128) corpus: * 🇬🇧 English: `newstest2019-ref.eng-US.txt` * 🇫🇷 French: `newstest2019-ref.fra.txt` * 🇪🇸 Spanish: `newstest2019-ref.spa.txt` * 🇵🇹 Portuguese: `newstest2019-ref.por.txt` * 🇩🇪 German: `newstest2019-ref.deu.txt` Using the English file, we select 300 groups of consecutive lines belonging to a same original document to form our multi-sentences source texts and obtain the target text translations accordingly. We define an `id` for each source-target pair as a hash of the ordered NTREX line indexes it comes from. We clean the source and target texts by removing elements in parentheses to make them better suited for natural speech. Each source text is then synthesized into 3 audio versions, each using a different TTS system and a different voice conditioning. We transcribe the synthesized audio using the [openai/whisper-large-v3](https://huggingface.co/openai/whisper-large-v3) Speech-To-Text model and check Word Error Rate with respect to the source texts to ensure that audio versions were correctly synthesized. We split the 3600 source/target pairs into balanced valid and test sets such that all pairs with the same target text stay in the same set i.e. we keep 150 different `id` for each language in each set. --- ## Citations If you use this dataset, please cite: ```bibtex @unpublished{hibikizero2026, title={Simultaneous Speech-to-Speech Translation Without Aligned Data}, author={Tom Labiausse and Romain Fabre and Yannick Estève and Alexandre Défossez and Neil Zeghidour}, note={Preprint}, year={2026}, url={https://arxiv.org/abs/2602.11072v1} } ``` **License:** CC BY-NC-SA 4.0