indicvoices_r

# IndicVoices-R: Multilingual, Multi-Speaker Speech Corpus for Indian TTS [](https://proceedings.neurips.cc/paper_files/paper/2024/file/7dfcaf4512bbf2a807a783b90afb6c09-Paper-Datasets_and_Benchmarks_Track.pdf) ## Dataset Summary **IndicVoices-R (IV-R)** is the largest multilingual Indian text-to-speech (TTS) dataset derived from an automatic speech recognition (ASR) dataset. It contains **1,704 hours** of high-quality speech from **10,496 speakers** across **22 Indian languages**. This dataset is designed to enhance the development of robust Indian TTS models by providing diverse speaker demographics, natural conversational speech, and high-quality audio samples. ## Key Features - **Comprehensive Coverage**: Includes **all 22 scheduled Indian languages**, ranging from **9 to 175 hours per language**. - **Speaker Diversity**: **10,496 speakers**, ensuring rich demographic and linguistic variation. - **High-Quality Samples**: Speech quality comparable to **gold-standard TTS datasets** (LJSpeech, LibriTTS, IndicTTS). - **Natural Speech Recordings**: **93.25% extempore speech**, leading to more expressive and natural-sounding synthesis. - **Zero-shot, Few-shot, and Many-shot Benchmarking**: Includes a **benchmark dataset** for evaluating speaker generalization capabilities of TTS models. ## Data Processing Pipeline The dataset was created by restoring and enhancing ASR-quality speech using: 1. **Demixing**: HTDemucs model to separate speech from background noise. 2. **Dereverberation**: VoiceFixer to reduce reverb and enhance speech clarity. 3. **Speech Enhancement**: DeepFilterNet3 to remove remaining artifacts. 4. **Filtering**: Samples filtered based on **speech clarity, SNR, pitch variation, and speaking rate**. 5. **Normalization**: Volume adjusted using PyDub for consistency. ## Dataset Format Each entry in the dataset includes: - **Audio file**: `.wav` format, 48 kHz sampling rate. - **Text transcript**: Available in both verbatim and normalized formats. - **Metadata JSON**: Includes speaker ID, gender, age group, duration, and language. ## Benchmarks & Evaluation IndicVoices-R includes a **benchmark suite** to evaluate **zero-shot, few-shot, and many-shot** TTS model performance across different demographics and speech styles. ### Key Evaluation Metrics: - **NORESQA-MOS**: Measures naturalness of synthesized speech. - **SNR & C50**: Assess speech clarity and reverberation levels. - **Speaker Similarity (S-SIM)**: Evaluates zero-shot speaker generalization. ## Usage & Applications IndicVoices-R can be used for: - **Training multilingual TTS models** with high speaker diversity. - **Improving zero-shot speaker generalization** for Indian languages. - **Building expressive and natural-sounding synthetic voices**. - **Evaluating TTS performance** with a standardized benchmark. ## License CC-BY-4.0 ## Acknowledgements This project was supported by Digital India Bhashini, the Ministry of Electronics and Information Technology (Government of India), EkStep Foundation, and Nilekani Philanthropies. Special thanks to CDAC Pune for access to the PARAM-Siddhi supercomputer, enabling our speech enhancement pipeline and model training. We also appreciate the unwavering support from the AI4Bharat team. ## Citation If you use IndicVoices-R, please cite the following paper: ``` @inproceedings{ai4bharat2024indicvoices_r, author = {Ashwin Sankar and Srija Anand and Praveen Srinivasa Varadhan and Sherry Thomas and Mehak Singal and Shridhar Kumar and Deovrat Mehendale and Aditi Krishana and Giri Raju and Mitesh M. Khapra}, editor = {Amir Globersons and Lester Mackey and Danielle Belgrave and Angela Fan and Ulrich Paquet and Jakub M. Tomczak and Cheng Zhang}, title = {IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian {TTS}}, booktitle = {Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024}, year = {2024}, url = {http://papers.nips.cc/paper\_files/paper/2024/hash/7dfcaf4512bbf2a807a783b90afb6c09-Abstract-Datasets\_and\_Benchmarks\_Track.html}, } ```