实时变声

Retrieval-based-Voice实时变声，使用top1检索替换输入源特征为训练集特征来杜绝音色泄漏即便在相对较差的显卡上也能快速训练使用少量数据进行训练也能得到较好结果(推荐至少收集10分钟底噪语音数据)可以通过模型融合来改变音色(借助ckpt处理选项卡中的ckpt-merge)简单易用的网页界面可调用UVR5模型来快速分离人声和伴奏使用最先进的人声音高提取算法InterSpeech2023-RMVPE根哑音问题。效果最好（显著地）但比crepe_full更快、资源占用更小

Retrieval-based real-time voice conversion: Uses top-1 retrieval to replace input source features with training set features to eliminate voice timbre leakage. It can be trained quickly even on relatively low-end GPUs, and achieves satisfactory results with only a small amount of training data (it is recommended to collect at least 10 minutes of speech data with background noise). The voice timbre can be adjusted via model fusion by utilizing the ckpt-merge function in the ckpt processing tab. It features an easy-to-use web interface, and supports invoking the UVR5 model for fast vocal and accompaniment separation. Adopting the state-of-the-art speech pitch extraction algorithm InterSpeech2023-RMVPE to address the silent segment problem, it delivers significantly better performance while being faster and more resource-efficient than crepe_full.

搜集汇总

背景与挑战

背景概述

该数据集基于检索的语音转换技术，旨在实现实时变声功能，使用高质量VCTK训练集以确保无版权问题。它支持快速训练和少量数据训练，并提供Web界面及多种硬件加速支持，适用于语音处理和音色转换任务。