DCASE 2024 Task 7 Dataset - Open Source

This dataset supports the development and evaluation of prompt-based generative algorithms for environmental sound synthesis. It is designed for the Sound Scene Synthesis task, which consists of generating realistic environmental sound scenes from textual descriptions. The dataset is a free and open version of the one used in the DCASE 2024 Task 7 challenge on sound scene synthesis. For a full description of the task and access to challenge results, please consult the official challenge page. An in-depth description of the challenge evaluation protocol and a detailed analysis of the results are available in [1]. Unlike the official challenge dataset, this version includes only audio sourced from Freesound and excludes any proprietary or private sound libraries. 📊 Dataset Overview The dataset includes 310 audio clips, each 4 seconds long, along with their corresponding text prompts. Unlike typical audio captioning datasets, both the prompts and audio scenes were manually crafted and edited. This enables a more controlled and quantifiable evaluation of generative models. Prompts follow a fixed structure: > (foreground sound source) with (background sound source) in the background Foreground sounds are action-based (e.g., a dog barking). They fall into six categories: - animal- vehicle- human- alarm- tool- entrance These are paired with five possible background categories: - crowd- traffic- water- birds- no background > Note: Foreground vehicle sounds are not paired with the traffic background to avoid redundancy. The no background category enables the evaluation of monophonic scenes with isolated foreground sources. The dataset is split into a developpement and an evaluation set: - Development Set: 60 audio–caption pairs (backgrounds: crowd, traffic, water)- Evaluation Set: 250 audio–caption pairs (backgrounds: crowd, traffic, water, birds, no background) 📁 Folder Structure Inside the DCASE-TASK7-2024-Open-Source/ folder: DCASE-TASK7-2024-Open-Source/├── dev/│ ├── audio/│ └── caption.csv├── eval/│ ├── audio/│ └── caption.csv - audio/: Contains the audios in wav format.- caption.csv: Provides corresponding prompts for each audio file. 📎 Citation If you use this dataset in your research, please cite it as: Tailleur, Modan; Lee, Junwon; Heller, Laurie; Choi, Keunwoo; McFee, Brian; Lagrange, Mathieu; Imoto, Keisuke; Okamoto, Yuki. DCASE 2024 Task 7 Dataset - Open Source. Zenodo, 2024. DOI: 10.5281/zenodo.15630417 @misc{dcase2024task7opensource, title = {DCASE 2024 Task 7 Dataset - Open Source}, author = {Tailleur, Modan and Lee, Junwon and Heller, Laurie and Choi, Keunwoo and McFee, Brian and Lagrange, Mathieu and Imoto, Keisuke and Okamoto, Yuki}, year = {2024}, publisher = {Zenodo}, doi = {10.5281/zenodo.15630417}} 📚 References [1] Lee, Junwon; Tailleur, Modan; Heller, Laurie M.; Choi, Keunwoo; Lagrange, Mathieu; McFee, Brian; Imoto, Keisuke; Okamoto, Yuki. Challenge on Sound Scene Synthesis: Evaluating Text-to-Audio Generation. In Audio Imagination: NeurIPS 2024 Workshop on AI-Driven Speech, Music, and Sound Generation, 2024.本数据集支持基于提示词的生成式算法在环境声音合成领域的开发与评估，其针对**声音场景合成（Sound Scene Synthesis）**任务设计，该任务旨在从文本描述生成逼真的环境声音场景。 本数据集是DCASE 2024任务7声音场景合成挑战赛所用数据集的免费开源版本。如需了解该任务的完整说明以及挑战赛结果，请查阅官方挑战赛页面；关于挑战赛评估协议的深度阐释与结果的详细分析，请参见文献[1]。与官方挑战赛数据集不同，本版本仅包含源自Freesound的音频素材，未使用任何专有或私有声音库。 📊 数据集概览 本数据集包含310段时长为4秒的音频片段及其对应的文本提示词。与典型的音频字幕（audio captioning）数据集不同，本数据集的提示词与音频场景均经过人工创作与编辑，可实现对生成式模型更具可控性与可量化的评估。 提示词遵循固定结构： >（前景声源），背景为（背景声源） 前景声源均为基于动作的声源（例如犬吠），共分为六大类别： - 动物类、交通工具类、人类活动类、警报类、工具类、出入口类 上述前景声源可与五类背景声源配对： - 人群声、交通声、水流声、鸟鸣声、无背景 > 注意：前景交通工具声源不与交通背景配对，以避免冗余；无背景类别可用于评估仅包含孤立前景声源的单声道场景。 本数据集分为开发集与评估集： - 开发集：共60组音频-字幕对，背景类型涵盖人群声、交通声、水流声 - 评估集：共250组音频-字幕对，背景类型涵盖人群声、交通声、水流声、鸟鸣声、无背景 📁 文件夹结构 在DCASE-TASK7-2024-Open-Source/文件夹内，结构如下： DCASE-TASK7-2024-Open-Source/ ├── dev/ │ ├── audio/ │ └── caption.csv ├── eval/ │ ├── audio/ │ └── caption.csv - audio/：存放wav格式的音频文件 - caption.csv：为每个音频文件提供对应的提示词 📎 引用说明 若您在研究中使用本数据集，请按以下格式引用： Tailleur, Modan; Lee, Junwon; Heller, Laurie; Choi, Keunwoo; McFee, Brian; Lagrange, Mathieu; Imoto, Keisuke; Okamoto, Yuki. DCASE 2024 Task 7 Dataset - Open Source. Zenodo, 2024. DOI: 10.5281/zenodo.15630417 @misc{dcase2024task7opensource, title = {DCASE 2024 Task 7 Dataset - Open Source}, author = {Tailleur, Modan and Lee, Junwon and Heller, Laurie and Choi, Keunwoo and McFee, Brian and Lagrange, Mathieu and Imoto, Keisuke and Okamoto, Yuki}, year = {2024}, publisher = {Zenodo}, doi = {10.5281/zenodo.15630417} } 📚 参考文献 [1] Lee, Junwon; Tailleur, Modan; Heller, Laurie M.; Choi, Keunwoo; Lagrange, Mathieu; McFee, Brian; Imoto, Keisuke; Okamoto, Yuki. 声音场景合成挑战赛：文本到音频生成（Text-to-Audio Generation）模型评估. 收录于《音频畅想：2024年神经信息处理系统大会AI驱动语音、音乐与声音生成研讨会》，2024年。