Data and code from: Motor origins of timbre in piano performance

Creativity in the arts, such as painting and musical performance, hinges on the ability to produce a wide spectrum of perceptual experiences. In music, it has long been believed that the timbre of tones can be altered by nuanced movements of performers. Previous studies have described relationships between fundamental elements of auditory perceptions (e.g., loudness, tempo) and physical movements (e.g., force, speed), but it remains unknown whether and how delicate features of perceptual experiences, such as tone timbre, are manipulated through dexterous motor skills. Here, we bridge this gap using a two-fold experimental approach. First, our listening test revealed that the timbral qualities pianists intended to express in piano playing were perceived as intended by both pianists and musically untrained individuals, with pianists showing a greater perceptual sensitivity to different timbres. Second, through a motor behavioral experiment using a non-contact, high-resolution sensing system, we identified five specific movement features in piano touch that were intricately linked to three categories of perceived timbre: weight, clarity, and brightness. Furthermore, the direct manipulation of a specific key movement feature resulted in systematic changes in perceived timbre, providing evidence for a causal relationship. The result indicates that pianists share common motor skills to modify perceived tone timbre by manipulating specific movement features. Our findings underscore the pivotal roles of subtle physical gestures in creating the rich timbral palette of piano tones, advancing our understanding of the intersection between motor control and artistic expression.

艺术领域的创造力，例如绘画与音乐表演，其核心在于生成丰富多维感知体验的能力。在音乐领域，学界长期以来认为演奏者的细微动作可改变音调音色（timbre）；过往研究已阐明听觉感知（auditory perceptions）的基础要素（如响度、节拍速度）与肢体动作（如施力、速率）之间的关联，但学界仍不清楚诸如音调音色这类感知体验的精细特征，是否可通过灵巧运动技能加以操控，以及具体的调控机制。本研究采用双重实验方法填补了这一研究空白：其一，听力测试结果显示，钢琴家在演奏中意图表达的音色特质，能够被钢琴家本人与无音乐训练经历的受试者精准感知，且钢琴家对不同音色的感知敏感性更强；其二，借助非接触式高分辨率传感系统开展运动行为实验，我们识别出钢琴触键过程中的五项特定运动特征，这些特征与三类感知音色——重量感、清晰度与明亮度——存在复杂关联，此外，直接操控某一特定按键运动特征会导致感知音色出现系统性变化，这为二者间的因果关系提供了实证依据。研究结果表明，钢琴家共享一套通用的运动技能，通过调控特定的运动特征来改变感知音色；本研究发现强调了细微肢体动作在塑造钢琴音色丰富调色板中的关键作用，加深了我们对运动控制与艺术表达交叉领域的理解。