m-a-p/Music-Instruct

---
license: cc-by-nc-4.0
---

# Music Instruct (MI) Dataset

This is the dataset used to train and evaluate the MusiLingo model.
This dataset contains Q&A pairs related
to individual musical compositions, specifically
tailored for open-ended music queries. It originates
from the music-caption pairs in the MusicCaps
dataset.
The MI dataset was created through prompt engineering and applying few-shot learning techniques
to GPT-4. More details on dataset generation can be found in our paper *[MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response
](https://arxiv.org/abs/2309.08730)*.

The resulting MI dataset consists of two versions:
v1 (short questions), with 27,540 Q&A pairs seeking comprehensive
details about musical snippets including but not limited to emotion, instrument, vocal track, tempo, and genre etc., often yielding concise one or two-sentence responses. In contrast, v2
comprises 32,953 Q&A pairs featuring more general questions about the musical pieces (long questions), resulting
in typically more extensive responses that serve as
paraphrased renditions of the original caption.

## Evaluation and dataset SPlittion
You can use all (or the long/short partition of) the Q\&A pairs of which audio is in the training split of AudioSet as MI training set and use the short QA and long QA with audio in evaluation split of AudioSet as two testingsets separately.

```
# training set
ds_mixed_train = MIDataset(processor, '/content/drive/MyDrive/music_data', split='train', question_type='all')
ds_long_train = MIDataset(processor, '/content/drive/MyDrive/music_data', split='train', question_type='long')
ds_short_train = MIDataset(processor, '/content/drive/MyDrive/music_data', split='train', question_type='short')
# testing set for short QA
ds_short = MIDataset(processor, '/content/drive/MyDrive/music_data', split='test', question_type='short')
# testing set for long QA
ds_long = MIDataset(processor, '/content/drive/MyDrive/music_data', split='test', question_type='long')
```

And the evaluation includes BLEU, METEOR, ROUGE, and Bert-Score.

## Citation
```
@article{deng2023musilingo,
title={MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response},
author={Deng, Zihao and Ma, Yinghao and Liu, Yudong and Guo, Rongchen and Zhang, Ge and Chen, Wenhu and Huang, Wenhao and Benetos, Emmanouil},
journal={arXiv preprint arXiv:2309.08730},
year={2023}
}
```

许可证：CC BY-NC 4.0

# 音乐指令（Music Instruct, MI）数据集

本数据集用于训练与评估MusiLingo模型。
本数据集包含与单首音乐作品相关的问答对（Q&A pairs），专为开放式音乐查询任务定制，数据源自MusicCaps数据集中的音乐-字幕配对样本。
MI数据集通过提示工程（prompt engineering），并将少样本学习（few-shot learning）技术应用于GPT-4构建而成。有关数据集生成的更多细节，请参阅我们的论文《MusiLingo：借助预训练语言模型搭建音乐与文本的桥梁，实现音乐字幕生成与查询响应》（https://arxiv.org/abs/2309.08730）。

最终生成的MI数据集包含两个版本：v1版本（短问题）包含27540组问答对，旨在获取音乐片段的全方位细节，涵盖情感、乐器、人声轨道、速度、流派等多个方面，回答通常为简洁的1至2个句子。与之相对，v2版本包含32953组问答对，围绕音乐作品提出更宽泛的问题（长问题），回答通常更为详尽，是对原音乐字幕的释义性转述。

## 评估与数据集划分
你可将音频属于AudioSet训练划分的所有问答对（或其中的长、短问答子集）作为MI训练集；同时将音频属于AudioSet评估划分的短问答与长问答分别作为两个测试集。

# training set
ds_mixed_train = MIDataset(processor, '/content/drive/MyDrive/music_data', split='train', question_type='all')
ds_long_train = MIDataset(processor, '/content/drive/MyDrive/music_data', split='train', question_type='long')
ds_short_train = MIDataset(processor, '/content/drive/MyDrive/music_data', split='train', question_type='short')
# testing set for short QA
ds_short = MIDataset(processor, '/content/drive/MyDrive/music_data', split='test', question_type='short')
# testing set for long QA
ds_long = MIDataset(processor, '/content/drive/MyDrive/music_data', split='test', question_type='long')

评估指标包含BLEU、METEOR、ROUGE与BERT-Score。

## 引用信息

@article{deng2023musilingo,
title={MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response},
author={Deng, Zihao and Ma, Yinghao and Liu, Yudong and Guo, Rongchen and Zhang, Ge and Chen, Wenhu and Huang, Wenhao and Benetos, Emmanouil},
journal={arXiv preprint arXiv:2309.08730},
year={2023}
}

原始信息汇总

Music Instruct (MI) 数据集

概述

Music Instruct (MI) 数据集用于训练和评估 MusiLingo 模型。该数据集包含与音乐作品相关的问答对，特别针对开放式音乐查询。数据集源自 MusicCaps 数据集中的音乐-标题对。MI 数据集通过提示工程和应用少量学习技术从 GPT-4 生成。

数据集版本

• v1 (短问题)：包含 27,540 个问答对，寻求关于音乐片段的详细信息，如情感、乐器、声轨、节奏和流派等，通常产生简短的一到两句话回答。
• v2 (长问题)：包含 32,953 个问答对，涉及更一般的音乐作品问题，通常产生更广泛的回答，作为原始标题的改写版本。

评估和数据集划分

数据集可以分为训练集和测试集：

• 训练集：使用 AudioSet 训练分割中的音频的所有（或长/短部分）问答对作为 MI 训练集。
• 测试集：使用 AudioSet 评估分割中的音频的短问答和长问答分别作为两个测试集。

训练集示例

python ds_mixed_train = MIDataset(processor, /content/drive/MyDrive/music_data, split=train, question_type=all) ds_long_train = MIDataset(processor, /content/drive/MyDrive/music_data, split=train, question_type=long) ds_short_train = MIDataset(processor, /content/drive/MyDrive/music_data, split=train, question_type=short)

测试集示例

python ds_short = MIDataset(processor, /content/drive/MyDrive/music_data, split=test, question_type=short) ds_long = MIDataset(processor, /content/drive/MyDrive/music_data, split=test, question_type=long)

评估指标

评估包括 BLEU、METEOR、ROUGE 和 Bert-Score。

引用

plaintext @article{deng2023musilingo, title={MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response}, author={Deng, Zihao and Ma, Yinghao and Liu, Yudong and Guo, Rongchen and Zhang, Ge and Chen, Wenhu and Huang, Wenhao and Benetos, Emmanouil}, journal={arXiv preprint arXiv:2309.08730}, year={2023} }