WavCaps

WavCaps是首个大规模弱标签音频字幕数据集，由萨里大学等机构创建，包含约40万音频片段及其配对字幕。数据来源于网络资源和声音事件检测数据集，通过ChatGPT自动过滤和转换原始描述为高质量字幕。该数据集适用于多种音频语言多模态学习任务，旨在解决现有音频语言数据集规模有限的问题，推动音频语言多模态研究的发展。WavCaps is the first large-scale weakly-labeled audio captioning dataset, created by institutions including the University of Surrey, containing approximately 400,000 audio clips and their paired captions. The data is sourced from web resources and sound event detection datasets, and original descriptions are automatically filtered and converted into high-quality captions via ChatGPT. This dataset is applicable to various audio-language multimodal learning tasks, aiming to address the limited scale issue of existing audio-language datasets and promote the development of audio-language multimodal research.

搜集汇总

背景与挑战

背景概述

WavCaps是一个大规模弱标签音频字幕数据集，包含约40万个音频片段及其配对的字幕，适用于音频-语言多模态学习研究。数据集通过ChatGPT处理原始描述，提高了字幕质量，支持多种下游任务如音频语言检索和自动音频字幕生成。