Tracking predictions in naturalistic music listening using MEG and computational models of music

This is the dataset acquired for the 2022 study by Kern, Heilbron, De Lange, & Spaak. Licensed CC-BY-4.0, to comply with attribution terms, please cite the related publication (peer-reviewed when available, preprint otherwise). Abstract for the preprint describing the study below:

Expectations shape our experience of music. However, the internal model upon which listeners form melodic expectations is still debated. Do expectations stem from Gestalt-like principles or statistical learning? If the latter, does long-term experience play an important role, or are short-term regularities sufficient? And finally, what length of context informs contextual expectations? To answer these questions, we presented human listeners with diverse naturalistic compositions from Western classical music, while recording neural activity using MEG. We quantified note-level melodic surprise and uncertainty using various computational models of music, including a state-of-the-art transformer neural network. A time-resolved regression analysis revealed that neural activity over fronto-temporal sensors tracked melodic surprise particularly around 200 ms and 300-500 ms after note onset. This neural surprise response was dissociated from sensory-acoustic and adaptation effects. Neural surprise was best predicted by computational models that incorporated long-term statistical learning - rather than by simple, Gestalt-like principles. Yet, intriguingly, the surprise reflected primarily short-range musical contexts of less than ten notes. We present a full replication of our novel MEG results in an openly available EEG dataset. Together, these results elucidate the internal model that shapes melodic predictions during naturalistic music listening.

本数据集源自Kern、Heilbron、De Lange与Spaak于2022年开展的研究，采用CC-BY-4.0开源许可协议。为遵守署名规范，请引用相关研究成果：若有同行评议版本则引用该正式版本，否则引用预印本。以下为描述本研究的预印本摘要：预期塑造了我们的音乐聆听体验。然而，听众形成旋律预期的内部模型仍存在诸多争议。这些旋律预期究竟源自格式塔（Gestalt）类原则，还是统计学习？若为统计学习路径，那么长期经验是否发挥关键作用，抑或仅依靠短期规律便已足够？最后，多长的语境能够驱动情境性预期？为解答上述问题，我们向人类听众呈现了多样化的西方古典音乐自然化作品，同时使用脑磁图（MEG）记录其神经活动。我们借助多种音乐计算模型（包括当前前沿的Transformer神经网络），量化了音符级别的旋律惊喜度与不确定性。时间分辨回归分析结果显示，额颞区域传感器记录的神经活动，在音符起始后约200毫秒及300-500毫秒时段，与旋律惊喜度的关联最为显著。该神经惊喜响应与感觉-声学效应及神经适应效应相互分离。相较于简单的格式塔类原则，纳入长期统计学习机制的计算模型对神经惊喜度的预测效果最佳。但值得注意的是，该惊喜度主要反映了长度不足10个音符的短程音乐语境。我们在一个公开可用的脑电图（EEG）数据集中，完整复现了这项创新性的脑磁图研究结果。综上，这些结果阐明了自然化音乐聆听过程中，塑造旋律预测的内部模型。