YodaLingua-preview

# YodaLingua-preview **YodaLingua** is a high-quality speech dataset designed for training text-to-speech (TTS) systems, ASR models, and any application requiring clean, well-aligned audio–text pairs. This version contains an equal number of samples from each language (100 samples per language). All audio clips are noise-reduced, normalized, and matched with accurate transcriptions. ## 🧾 Dataset Overview (full version) | Property | Value | |---------|-------| | **Total clips** | 1,839,325 audio–transcription pairs | | **Total duration** | ~5000 hours | | **Speakers** | 75,793 distinct speakers | | **Audio format** | MP3 • mono • 24 kHz • 16-bit | | **License** | Permissive — commercial use allowed | --- ## Data Fields Each entry in the dataset contains the following fields: | Field | Description | |------|------------| | `__key__` | Unique identifier for each sample. | | `audio` | Path to the audio file associated with the sample (MP3 format). | | `text` | Ground-truth transcription of the audio segment. | | `language` | Language code following ISO 639 standards. | | `speaker_id` | Unique identifier assigned to each speaker. Multiple audio can share the same speaker ID. | | `dnsmos` | DNSMOS P.835 Overall (OVRL) score estimating perceptual speech quality; higher values indicate cleaner and more intelligible audio. | --- ## License This dataset is released under the [Creative Commons Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/) license. It can be used for commercial and non-commercial purposes and attribution is not required. --- ## 🌍 Full Versions To access the complete dataset, go to the collection: 👉 https://huggingface.co/collections/Thomcles/yodalingua --- We apply a multi-stage pipeline to ensure maximum data quality: ### **1. Standardization** - Convert to WAV - Mono channel - Resample to **24 kHz** - **16-bit** sample width - Normalize to **–20 dBFS** (with volume correction between –3 and +3 dB) ### **2. Noise Reduction** Advanced denoising applied to improve clarity and remove background artifacts. ### **3. Speaker Diarization** Segment long recordings by speaker to improve diversity and ensure speaker-consistent utterances. ### **4. Voice Activity Detection (VAD)** Merge consecutive VAD segments from the same speaker into clean utterances of **3–30 s**. ### **5. Transcription** State-of-the-art ASR models produce accurate text transcripts. ### **6. Quality Filtering** Clips are filtered using **DNSMOS P.835 OVRL**; only samples with a score **> 2.4** are retained. ## Contact e-mail : [[e-mail]]([e-mail])