emolia_fitered_nano_codec_21_dataset

# Emolia · Filtered · NanoCodec (FSQ) Tokens A cleaned, pre-tokenized version of [**laion/Emolia**](https://huggingface.co/datasets/laion/Emolia) prepared for **text-to-speech (TTS)** training. The pipeline is two steps: 1. **Quality filtering** with the open-source [`audio_filter`](https://github.com/KaniTTS-research-team/audio_filter) tool — this removes the dirtiest recordings (noise, clipping, band-limiting, robotic artifacts, overlapping speakers), which matters a lot for TTS quality. 2. **Discrete audio tokenization** with NVIDIA [`nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps`](https://huggingface.co/nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps) (an **FSQ** neural audio codec) — every waveform is replaced by 8 layers of integer codes. The result is a compact, ready-to-train dataset: **text + speaker + language + audio codec tokens**. No raw audio is redistributed here — only discrete codec indices. - **Examples:** 5,397,323 (single `train` split) - **On-disk size:** ~68.9 GB (Apache Arrow, 138 shards) - **Audio codec:** NanoCodec, 22 kHz, 1.89 kbps, 21.5 frames/sec, 8 FSQ codebooks - **Source dataset:** [laion/Emolia](https://huggingface.co/datasets/laion/Emolia) (CC-BY-4.0) --- ## Dataset structure Each row is one utterance. There is no audio blob — the audio is encoded as 8 parallel streams of NanoCodec indices (`nano_layer_1` … `nano_layer_8`), each of length `encoded_len`. | Column | Type | Description | | --------------- | ------------- | ----------------------------------------------------------------- | | `id` | `string` | Unique utterance id (carried over from Emolia) | | `text` | `string` | Transcript of the utterance | | `speaker` | `string` | Speaker id | | `language` | `string` | Language code (`en`, `zh`, `de`, `fr`, `ja`, `ko`) | | `nano_layer_1` | `list<int64>` | NanoCodec FSQ codes, codebook 1 — length `encoded_len` | | `nano_layer_2` | `list<int64>` | NanoCodec FSQ codes, codebook 2 | | `nano_layer_3` | `list<int64>` | NanoCodec FSQ codes, codebook 3 | | `nano_layer_4` | `list<int64>` | NanoCodec FSQ codes, codebook 4 | | `nano_layer_5` | `list<int64>` | NanoCodec FSQ codes, codebook 5 | | `nano_layer_6` | `list<int64>` | NanoCodec FSQ codes, codebook 6 | | `nano_layer_7` | `list<int64>` | NanoCodec FSQ codes, codebook 7 | | `nano_layer_8` | `list<int64>` | NanoCodec FSQ codes, codebook 8 | | `encoded_len` | `int64` | Number of codec frames (≈ duration_seconds × 21.5) | > The 8 `nano_layer_*` columns are aligned frame-by-frame: index `t` of every layer > describes the same 1/21.5 s ≈ 46 ms audio frame. --- ## Usage ### Load the dataset ```python from datasets import load_dataset # Streaming is recommended — the dataset is large. ds = load_dataset("nineninesix/emolia_filtered_nano_codec_21_dataset", split="train", streaming=True) sample = next(iter(ds)) print(sample["text"], sample["language"], sample["speaker"]) print("frames:", sample["encoded_len"]) print("codebook 1:", sample["nano_layer_1"][:10]) ``` ### Reconstruct audio from the tokens The codes can be decoded back to a 22 kHz waveform with the matching NeMo NanoCodec model. ```python import torch from nemo.collections.tts.models import AudioCodecModel codec = AudioCodecModel.from_pretrained("nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps") codec.eval() # Stack the 8 layers into (Batch=1, Codebooks=8, Time) layers = [sample[f"nano_layer_{i}"] for i in range(1, 9)] tokens = torch.tensor(layers, dtype=torch.long).unsqueeze(0) # (1, 8, T) tokens_len = torch.tensor([sample["encoded_len"]], dtype=torch.long) # (1,) with torch.no_grad(): audio, audio_len = codec.decode(tokens=tokens, tokens_len=tokens_len) # audio -> 22 kHz waveform, save with soundfile/torchaudio ``` --- ## How this dataset was built ### 1. Quality filtering — [`audio_filter`](https://github.com/KaniTTS-research-team/audio_filter) We ran every Emolia recording through the open-source [`audio_filter`](https://github.com/KaniTTS-research-team/audio_filter) pipeline and kept only the clean ("good") audio. The tool analyzes acoustic / spectral characteristics and assigns each file a verdict of `good`, `uncertain`, or `bad` in two sequential stages: - **Acoustic quality screening** — rejects noisy, clipped, robotic, or bandwidth-limited recordings using spectral degradation metrics. Two models are used depending on the audio bandwidth: - **V1** for narrowband audio (≤ 24 kHz), scoring 12 acoustic metrics; - **V2** for wideband audio (> 24 kHz), scoring 34 acoustic metrics with loudness normalization. - **Speaker-overlap detection** — a Pyannote segmentation model flags files that contain simultaneous speech from multiple speakers; this runs only on files that already passed the quality stage. Only recordings that passed both stages ("good") were retained for tokenization. This step is what removes the "dirtiest" audio and is critical for downstream TTS quality. > Reproduce the filtering with the exact tool and thresholds documented in the repo: > <https://github.com/KaniTTS-research-team/audio_filter> ### 2. Tokenization — NVIDIA NanoCodec (FSQ) The surviving clean audio was encoded with [`nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps`](https://huggingface.co/nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps), a **Finite Scalar Quantization (FSQ)** neural audio codec: - **Sample rate:** 22 kHz - **Bitrate:** 1.89 kbps - **Frame rate:** 21.5 frames/second - **Codebooks:** 8 (stored as `nano_layer_1` … `nano_layer_8`) Each waveform becomes 8 aligned integer streams plus `encoded_len` (the number of frames). Raw audio is **not** included — only these discrete indices, which keeps the dataset compact and avoids redistributing the source audio. --- ## Provenance & derivation This dataset is a **derivative** of [**laion/Emolia**](https://huggingface.co/datasets/laion/Emolia), an emotion-enriched multilingual speech dataset built on top of **Emilia** (and its YODAS subset). We did **not** change the transcripts, speaker ids, or language labels — we only: 1. **Filtered out** low-quality / overlapping-speaker recordings, and 2. **Replaced the audio waveforms** with NanoCodec (FSQ) discrete tokens. For details about the original annotations, emotion labels, and speaker embeddings, see the [Emolia dataset card](https://huggingface.co/datasets/laion/Emolia). --- ## Licensing & attribution - **This packaging (README, filtering/tokenization pipeline, and the code representation of the data) is released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).** - The **underlying speech content is derived from [laion/Emolia](https://huggingface.co/datasets/laion/Emolia), which is distributed under [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/).** Attribution to LAION / the Emolia and Emilia authors is required, and you must comply with CC-BY-4.0 (and the original Emilia terms) when using the audio-derived content in this dataset. By using this dataset you agree to respect both the Apache-2.0 terms of this repository and the CC-BY-4.0 terms of the upstream Emolia data. --- ## Citation If you use this dataset, please cite the upstream Emolia / Emilia work and this repository. ```bibtex @misc{emolia_filtered_nanocodec, title = {Emolia · Filtered · NanoCodec (FSQ) Tokens}, author = {Nineninesix, Inc.}, year = {2026}, howpublished = {\url{https://huggingface.co/datasets/nineninesix/emolia_filtered_nano_codec_21_dataset}}, note = {Filtered and NanoCodec-tokenized derivative of laion/Emolia} } ``` Please also cite the source dataset [laion/Emolia](https://huggingface.co/datasets/laion/Emolia) and the NVIDIA NanoCodec model. --- ## Acknowledgements - [LAION](https://laion.ai/) and the Emolia / Emilia authors for the source dataset. - [NVIDIA NeMo](https://github.com/NVIDIA/NeMo) for the NanoCodec (FSQ) audio codec. - [`audio_filter`](https://github.com/KaniTTS-research-team/audio_filter) for the quality-filtering pipeline.