muchomusic

Dataset Summary
MuChoMusic is a benchmark designed to evaluate music understanding in multimodal audio-language models (Audio LLMs). The dataset comprises 1,187 multiple-choice questions created from 644 music tracks, sourced from two publicly available music datasets: MusicCaps and the Song Describer Dataset (SDD). The questions test knowledge and reasoning abilities across dimensions such as music theory, cultural context, and functional applications. All questions and answers have been validated by human annotators to ensure high-quality evaluation.\
\
This dataset is a re-upload of [mulab-mir/muchomusic](https://huggingface.co/datasets/mulab-mir/muchomusic) intended for use in the [lmms-eval](https://github.com/EvolvingLMMs-Lab/lmms-eval) framework, a suite of benchmarks for evaluating large multimodal models.\
This dataset follows the licensing terms specified in the original [paper](https://arxiv.org/pdf/2408.01337), which is under the Creative Commons Attribution 4.0 License (CC BY 4.0).

数据集概览
MuChoMusic 是一款用于评估多模态音频-语言模型（Audio LLMs）音乐理解能力的基准测试集。该数据集包含基于644首音乐曲目构建的1187道选择题，其数据源来自两个公开音乐数据集：MusicCaps 与歌曲描述符数据集（Song Describer Dataset，SDD）。题目从音乐理论、文化背景、功能应用等多个维度，考察模型的知识储备与推理能力。所有题目与答案均经过人工标注者校验，以保障评估的高质量。

本数据集为[mulab-mir/muchomusic](https://huggingface.co/datasets/mulab-mir/muchomusic)的重新上传版本，旨在适配[lmms-eval](https://github.com/EvolvingLMMs-Lab/lmms-eval)框架——该框架是一套用于评估大型多模态模型的基准测试套件。
本数据集遵循原[论文](https://arxiv.org/pdf/2408.01337)中规定的许可条款，采用知识共享署名4.0国际许可协议（CC BY 4.0）。