JukeBox

我们介绍了Jukebox，这是一种在原始音频域中通过唱歌生成音乐的模型。我们使用多尺度vq-vae来处理原始音频的长上下文，将其压缩为离散代码，并使用自回归转换器对其进行建模。我们表明，大规模的组合模型可以生成高保真和多样化的歌曲，其连贯性长达几分钟。我们可以根据艺术家和流派来控制音乐和人声风格，并根据不一致的歌词来使演唱更加可控。我们正在发布数千个非樱桃挑选的样本，以及模型权重和代码。

We introduce Jukebox, a model for generating music with singing in the raw audio domain. We utilize multi-scale VQ-VAE to process long-span raw audio contexts, compress them into discrete codes, and model these codes using autoregressive Transformers. We demonstrate that large-scale composite models can generate high-fidelity and diverse songs with coherent structure lasting up to several minutes. Our framework enables control over musical and vocal styles by specifying artist and genre, and enhances the controllability of singing by conditioning on inconsistent lyrics. We are releasing thousands of non-cherry-picked samples, along with the model weights and source code.