Measuring embodied analogy-making in singing performances: insights from an OpenPose study

Kinesemiotics studies how we create meaning with their bodies by making projections onto space. This type of embodied analogy-making is fundamental to how we conceptualize experiences and the embodiment hypothesis postulates that conventional meanings are reminiscent of this sense-making process. This entails that conventions reflect the way we conceptualize the world. Interestingly, in musical interactions, people rely both on widely established conventions and very local analogy-making, which suggests that both processes interact simultaneously. Accordingly, the present study aims at empirically investigating how conventionality and situatedness interact in shaping spatial conceptualizations of pitch as a property of music. In many languages around the world, pitch is conceptualized in terms of spatial metaphors (e.g. ‘low’ and ‘high’ tones in Dutch) and this convention can be enacted using bodily projections in space (e.g. by referring to low and high locations). An experiment was conducted, in which participants (native speakers of Dutch) had to listen to and reproduce melodies using vocalizations. We used OpenPose to track the participants’ head movements during both listening and the singing phases. One group (n=19) could read staff notation, while the other group (n=19) could not, indicating less familiarity with musical conventions. The main hypothesis was that participants having less experience with musical conventions would rely more on embodied analogy-making, showing more systematic vertical head movements that are congruent with the pitch contours (i.e. falling and rising) as compared to participants with more experience. This hypothesis was confirmed, and the result entails that a lesser degree of formal musical education corresponds to a need for a more elaborate sense-making process in a particular performance context, challenging recent studies on the role of conventionality in conceptualizations.