A machine learning approach to ornamentation modeling and synthesis in jazz guitar

We present a machine learning approach to automatically generate expressive (ornamented) jazz performances from un-expressive music scores. Features extracted from the scores and the corresponding audio recordings performed by a professional guitarist were used to train computational models for predicting melody ornamentation. As a first step, several machine learning techniques were explored to induce regression models for timing, onset, and dynamics (i.e. note duration and energy) transformations, and an ornamentation model for classifying notes as ornamented or non-ornamented. In a second step, the most suitable ornament for predicted ornamented notes was selected based on note context similarity. Finally, concatenative synthesis was used to automatically synthesize expressive performances of new pieces using the induced models. Supplemental online material for this article containing musical examples of the automatically generated ornamented pieces can be accessed at doi: 10.1080/17459737.2016.1207814 and https://soundcloud.com/machine-learning-and-jazz. In the Online Supplement we present an example of the musical piece Yesterdays by Jerome Kern, which was modeled using our methodology for expressive music performance in jazz guitar.

本研究提出一种机器学习（machine learning）方法，可从无表现力的乐谱自动生成富有表现力（带装饰音）的爵士演奏。研究提取了乐谱与专业吉他手演奏的对应音频录音中的特征，用于训练计算模型以预测旋律装饰音（melody ornamentation）。第一步，我们探索了多种机器学习技术，以构建用于时值（timing）、起音（onset）与力度（dynamics，即音符时值与能量）转换的回归模型，以及用于将音符分类为装饰音与非装饰音的装饰音分类模型。第二步，基于音符上下文相似度（note context similarity），为预测出的装饰音音符选取最适配的装饰音型。最后，采用拼接合成（concatenative synthesis）技术，利用所构建的模型自动合成新乐曲的富有表现力的演奏版本。本文的在线补充材料包含自动生成的带装饰音乐曲的音乐示例，可通过数字对象标识符（DOI）10.1080/17459737.2016.1207814 与网址 https://soundcloud.com/machine-learning-and-jazz 获取。在线补充材料中还展示了由杰罗姆·克恩（Jerome Kern）创作的乐曲《Yesterdays》的示例，该乐曲通过我们的爵士吉他表现力演奏方法完成了建模。