Emilia

Recently, speech generation models have made significant progress by using large-scale training data. However, the research community struggle to produce highly spontaneous and human-like speech due to the lack of large-scale, diverse, and spontaneous speech data. This paper present Emilia, the first multilingual speech generation dataset from in-the-wild speech data, and Emilia-Pipe, the first open-source preprocessing pipeline designed to transform in-the-wild speech data into high-quality training data with annotations for speech generation. Emilia starts with over 101k hours of speech in six languages and features diverse speech with varied speaking styles. To facilitate the scale-up of Emilia, the open-source pipeline Emilia-Pipe can process one hour of raw speech data ready for model training in a few mins, which enables the research community to collaborate on large-scale speech generation research. Experimental results validate the effectiveness of Emilia.

近年来，语音生成模型（speech generation model）依托大规模训练数据取得了显著进展。然而，由于缺乏大规模、多样化且自然自发的语音数据，研究界难以生成高度自然且类人的语音。本文提出了Emilia——首个基于野外采集语音数据（in-the-wild speech data）的多语言语音生成数据集，以及Emilia-Pipe——首个专为语音生成任务设计的开源预处理流水线（preprocessing pipeline），该流水线可将野外采集语音数据转换为带有语音生成相关标注的高质量训练数据。Emilia以覆盖六种语言的超10.1万小时语音语料为基础，涵盖了多种不同的口语表达风格。为推动Emilia的规模化应用，开源预处理流水线Emilia-Pipe可在数分钟内完成单小时原始语音数据的处理，使其可直接用于模型训练，这使得研究界能够携手开展大规模语音生成相关研究。实验结果验证了Emilia的有效性。

搜集汇总

背景与挑战

背景概述

Emilia是一个大规模、多语言的语音生成数据集，包含超过101k小时的语音数据，覆盖英语、中文、德语、法语、日语和韩语六种语言，数据来源于多样化的互联网视频和播客。该数据集公开可用，并提供了名为Emilia-Pipe的预处理管道，支持语音生成研究。