CSD: Children's Song Dataset for Singing Voice Research

Children's Song Dataset is open source dataset for singing voice research. This dataset contains 50 Korean and 50 English songs sung by one Korean female professional pop singer. Each song is recorded in two separate keys resulting in a total of 200 audio recordings. Each audio recording is paired with a MIDI transcription and lyrics annotations in both grapheme-level and phoneme-level. <strong>Dataset Structure</strong> The entire data splits into Korean and English and each language splits into 'wav', 'mid', 'lyric', 'txt' and 'csv' folders. Each song has the identical file name for each format. Each format represents following information. Additional information like original song name, tempo and time signature for each song can be found in 'metadata.json'. 'wav': Vocal recordings in 44.1kHz 16bit wav format 'mid': Score information in MIDI format 'lyric': Lyric information in grapheme-level 'txt': Lyric information in syllable and phoneme-level 'csv': Note onsets and offsets and syllable timings in comma-separated value (CSV) format <strong>Vocal Recording</strong> While recording vocals, the singer sang along with the background music tracks. She deliberately rendered the singing in a “plain” style refraining from expressive singing skills. The recording took place in a dedicated soundproof room. Singer recorded three to four takes for each song and the best parts are combined into a single audio track. Two identical songs with different keys are discriminated by character 'a' and 'b' at the end of a filename. <strong>MIDI Transcription</strong> The MIDI data consists of monophonic notes. Each note contains onset and offset times which were manually fine-tuned along with the corresponding syllable. MIDI notes do not include any expression data or control change messages because those parameters can be ambiguous to define for singing voice. Singing voice is an highly expressive sound and it is hard to define precise onset timings and pitches. We assumed one syllable matches with one MIDI note and made the following criteria to represent various expressions in singing voice. A piano sound is used as a reference tone for the annotated MIDI to ensure the alignment with vocal. The rising pitch at the beginning of a note is included within a single note. The end of syllable is treated as the offset of a note. The breathing sound during short pauses is not treated as note onset or offset. Vibrations are treated as a single sustaining note. If a syllable is rendered with several different pitches, we annotated them as separate notes. <strong>Lyric Annotation</strong> Text files in the 'lyric' folder contains raw text for corresponding audio and the 'txt' folder contains phoneme-level lyric representation. The phoneme-level lyric representation is annotated in a special text format. Phonemes in a syllable are tied with underbar('_') and syllables are separated with space(' '). Each phonemes are annotated based on the international phonetic alphabet (IPA) and romanized symbols are used to annotate IPA symbols. You can find romanized IPA symbols and more detailed information in this repository. <strong>License</strong> This dataset was created by the KAIST Music and Audio Computing Lab under Industrial Technology Innovation Program (No. 10080667, Development of conversational speech synthesis technology to express emotion and personality of robots through sound source diversification) supported support by the Ministry of Trade, Industry &amp; Energy (MOTIE, Korea). CSD is released under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).<strong> It is provided primarily for research purposes and it is prohibited to be used for commercial purposes. When sharing your result based on CSD, any act that defames the original singer is strictly prohibited.</strong> For more details, we refer to the following publication. We would highly appreciate if publications partly based on CSD quote the following publication: Choi, S., Kim, W., Park, S., Yong, S., &amp; Nam, J. (2020). Children’s Song Dataset for Singing Voice Research. 21th International Society for Music Information Retrieval Conference (ISMIR). We are interested in knowing if you find CSD useful. If you use CSD please email us at [e-mail] and tell us about your research.儿童歌曲数据集（Children's Song Dataset，以下简称CSD）是面向歌声研究的开源数据集。本数据集包含50首韩语歌曲与50首英语歌曲，均由一名韩国女性职业流行歌手演唱。每首歌曲以两种不同调式录制，总计得到200条音频录音。每条音频录音均配有MIDI转录文件，以及字素级与音素级的歌词标注。 <strong>数据集结构</strong> 全部数据按语种分为韩语与英语两大分支，每个语种分支下均设有'wav'、'mid'、'lyric'、'txt'和'csv'五个文件夹。同一首歌曲的各格式文件命名完全一致，各格式文件分别对应如下信息。每首歌曲的原始歌名、速度与拍号等额外信息可在'metadata.json'文件中查看： 'wav'：44.1kHz采样率、16位量化的人声录音文件 'mid'：MIDI格式的乐谱信息 'lyric'：字素级歌词信息 'txt'：音节级与音素级歌词信息 'csv'：以逗号分隔值（Comma-Separated Value, CSV）格式存储的音符起始终止时间与音节时序信息 <strong>人声录制</strong> 录制人声时，歌手伴随背景音乐轨进行演唱。她刻意采用“平实”的演唱风格，避免使用任何富有表现力的演唱技巧。录制工作在专业隔音棚内完成。歌手为每首歌曲录制3至4个演唱片段，最终选取最优片段拼接为单条音轨。两首调式不同的同名歌曲可通过文件名末尾的字符'a'和'b'加以区分。 <strong>MIDI转录</strong> MIDI数据由单音音符构成。每个音符均包含起始终止时间，该时间已与对应音节进行人工微调对齐。MIDI音符不包含任何表情数据或控制变更消息，因为对于人声演唱而言，这类参数的定义往往存在歧义。人声是极具表现力的声源，精准定义其起始时间与音高存在较大难度。我们假设一个音节对应一个MIDI音符，并制定了如下标准以表征人声演唱中的各类表情变化： 1. 以钢琴音色作为标注MIDI的参考音，确保与人声对齐精度 2. 将音符起始处的音高上升过程包含在单个音符范围内 3. 将音节结束时刻视为音符的终止时刻 4. 短暂停顿期间的呼吸声不视为音符的起始或终止点 5. 将颤音视为单个持续音符 6. 若一个音节使用多个不同音高演唱，则将其标注为多个独立音符 <strong>歌词标注</strong> 'lyric'文件夹中的文本文件存储对应音频的原始歌词文本，'txt'文件夹则存储音素级歌词表示。音素级歌词标注采用特殊文本格式：音节内的音素以下划线'_'连接，音节间以空格' '分隔。所有音素均基于国际音标（International Phonetic Alphabet, IPA）标注，并用罗马化符号替代IPA符号书写。本仓库提供罗马化IPA符号表与更多详细说明。 <strong>授权协议</strong> 本数据集由韩国科学技术院（KAIST）音乐与音频计算实验室创建，得到韩国贸易、工业与能源部（Ministry of Trade, Industry & Energy, MOTIE）产业技术创新项目（项目编号：10080667，《通过音源多样化实现机器人情感与个性表达的对话式语音合成技术开发》）资助。CSD采用知识共享署名-非商业性使用-相同方式共享4.0国际许可协议（Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International, CC BY-NC-SA 4.0）发布。本数据集仅用于学术研究，严禁用于商业用途。基于CSD分享研究成果时，严禁出现任何诋毁原歌手的行为。 如需了解更多细节，请参考如下论文。若您的研究成果部分基于CSD，恳请引用该文献：Choi, S., Kim, W., Park, S., Yong, S., & Nam, J. (2020). 面向歌声研究的儿童歌曲数据集. 第21届国际音乐信息检索大会（21th International Society for Music Information Retrieval Conference, ISMIR）。 我们十分期待了解您对CSD的使用反馈。若您使用本数据集，请发送邮件至[e-mail]，向我们介绍您的研究工作。