expresso-conversational-en-nano-codec-dataset

# Expresso Conversational EN Nano-Codec Dataset This dataset is built upon the [Expresso conversational dataset](https://huggingface.co/datasets/nytopop/expresso-conversational) and re-encoded using NVIDIA’s [NeMo Audio Codec](https://huggingface.co/nvidia/nemo-nano-codec-22khz-0.6kbps-12.5fps) into **nano audio tokens**. It is designed for **fine-tuning multimodal LLMs** and **speech systems (TTS/ASR)** that rely on codec-based audio token representations. --- ## Dataset Structure - **text**: transcription of the utterance. - **speaker**: speaker identifier (string). - **nano_layer_1 … nano_layer_4**: tokenized audio representations from the NVIDIA NeMo Nano Codec (4-layer quantization). - **encoded_len**: sequence length of encoded audio tokens. --- ## Use Cases - **Fine-tuning TTS** models with codec-based speech tokens. - **Training ASR** systems that operate on discrete audio units. - **Multimodal LLM adaptation**, where text and audio tokens are combined. This format makes it easier to build compact and efficient speech-enabled LLMs. --- ## Example ```python from datasets import load_dataset ds = load_dataset("nineninesix/expresso-conversational-en-nano-codec-dataset", split="train") print(ds[0]["text"]) # "Ribbit Nice to meet you, Stephen." print(ds[0]["nano_layer_1"][:10]) # [1633, 2685, 3825, 1392, ...] ```` --- ## Credits * Original data: [Expresso dataset](https://huggingface.co/datasets/nytopop/expresso-conversational). * Audio codec tokenization: [NVIDIA NeMo Codec](https://huggingface.co/nvidia/nemo-nano-codec-22khz-0.6kbps-12.5fps).