MusiQAl: Music Question-Answering through Audio-Video Fusion

What are the most relevant questions to ask a machine about a music performance? MusiQAl is a dataset created and annotated by a research team at RITMO for the task of Music Question Answering (MQA).

Purpose The purpose of creating MusiQal is to address significant limitations in existing MQA datasets, which often focus only on instrument performance and overlook important qualitative characteristics such as interaction, performance energy, and audience engagement. MusiQAl bridges existing gaps by catering to three performance types: instrument, singing, and dancing. It introduces a new set of questions that capture nuanced aspects of music performance, inspired by music cognition theories and advancements in machine perception of music.

Utility Audio-video and question-answer pairs are all available in this repository. Extracted features are also provided, as well as the code for constructing and evaluating the dataset in SOTA models.