Data and code from: Motor origins of timbre in piano performance

Creativity in the arts, such as painting and musical performance, hinges
on the ability to produce a wide spectrum of perceptual experiences. In
music, it has long been believed that the timbre of tones can be altered
by nuanced movements of performers. Previous studies have described
relationships between fundamental elements of auditory perceptions (e.g.,
loudness, tempo) and physical movements (e.g., force, speed), but it
remains unknown whether and how delicate features of perceptual
experiences, such as tone timbre, are manipulated through dexterous motor
skills. Here, we bridge this gap using a two-fold experimental approach.
First, our listening test revealed that the timbral qualities pianists
intended to express in piano playing were perceived as intended by both
pianists and musically untrained individuals, with pianists showing a
greater perceptual sensitivity to different timbres. Second, through a
motor behavioral experiment using a non-contact, high-resolution sensing
system, we identified five specific movement features in piano touch that
were intricately linked to three categories of perceived timbre: weight,
clarity, and brightness. Furthermore, the direct manipulation of a
specific key movement feature resulted in systematic changes in perceived
timbre, providing evidence for a causal relationship. The result indicates
that pianists share common motor skills to modify perceived tone timbre by
manipulating specific movement features. Our findings underscore the
pivotal roles of subtle physical gestures in creating the rich timbral
palette of piano tones, advancing our understanding of the intersection
between motor control and artistic expression.