Data from: Trade-offs in Coordination Strategies for Networked Jazz Performances

This dataset is associated with the paper “Trade-offs in Coordination Strategies for Networked Jazz Performances” and includes recordings of improvisations by jazz duos over a network. The preprint is accessible on PsyArXiv. Introduction: This dataset includes data from approximately four hours of live, improvised musical duo performances over a simulated network environment collected in Cambridge, United Kingdom between April-July 2022 as part of a doctoral research project. Data includes audio and video recordings of 130 individual performances, biometric data, and subjective evaluations and comments from the musicians. The primary aim of the project was to collect data via a novel performance capture and manipulation system for use in the empirical modelling of ensemble coordination strategies during networked music-making. This analysis is reported in Cheston, Cross, and Harrison (2023), "Trade-offs in Coordination Strategies for Networked Jazz Performances". Please refer to this publication for full details on the data collection procedure. Our codebook is hosted on GitHub and, in conjunction with this dataset, can be used to reproduce the analysis contained in the article. The ten musicians shown in these recordings were recruited for their expertise in jazz improvisation. They were grouped into five duos consisting each of one pianist and drummer, with no musician performing in more than one duo. Participants were instructed to improvise together over a standard twelve-bar blues musical structure, but following a formula which required them to provide a clear and unambiguous pulse of continuous quarter notes. Varying amounts of network latency and jitter were simulated for each performance, consisting respectively of the minimum amount of delay applied to the live feedback a musician heard from their partner and the degree that this delay varied. The amount of latency and jitter applied to the performance is summarised in the file or directory name for each performance and is described in detail in the above publication. Note that latency and jitter conditions were presented in a random order for each duo. Data collected includes: audio recordings for each performance, with and without delay, collected via direct line-in (MIDI, WAV). video recordings, collected via high-quality webcams (MKV, AVI). streams of the quarter note pulse provided by each musician in a performance (MIDI). muxed audio-visual recordings of both participants in each performance (MP4) accelerometer and photoplethysmography streams, collected from arm-worn devices (TXT, duos 3-5 only) questionnaire responses from performers, evaluating each condition (XLSX) ratings of performance quality from an unbiased sample of listeners, collected during an online perceptual study (CSV) Repository structure: NB: please see this section of the code documentation website for a full description of how to recreate the analyses and models created in the paper. The files data.zip and data.z0* contain all data collected from the study, APART from the perceptual study stimuli & results. To open these files, download the data.zip file and all the corresponding volumes ending in .z0 and open the data.zip file using a tool for opening multi-part zip files, such as WinRAR. Do not try to open the files ending in .z0, otherwise you may get a message about the data being corrupted. Inside data.zip, you'll see the following folders and files: avmanip_output: the raw MIDI, audio, and video output from each performance the subfolders are organised with a single folder per participant duo, experimental block, and condition. avmanip_output\trial_1\Block 1\Condition 1 - 23 05 relates to the performance of the first duo of participants in the first session of the experiment, in the first condition they encountered, with 23ms of latency and 0.5x jitter. midi_bpm_cleaning: the cleaned MIDI files (quarter note onset positions) the subfolders are organised similarly to the avmanip_output folder, using the same conventions. muxed_performances: the combined audio-video .mp4 files from each performance these files are labelled in the format: duo_session_latency_jitter_keysfmt_drumsfmt. muxed_performances\kdelay_ddelay\d1_s1_l23_j00_kdelay_ddelay.mp4 relates to the performance of the first duo of participants in the first session of the experiment, in the first condition they encountered, with 23ms of latency and 0.5x jitter, and with latency and jitter applied to both keys and drummer. for more information on recreating these videos, see the linked section of the code documentation website. questionnaire_anonymized: the anonymized questionnaire responses given by participants, also contained in the supplementary material of the associated paper (see preprint). Alongside data.zip and the data.z0* archives, there are two further loose files, Database View Participant - Dashboard.csv, Database View SuccessTrial - Dashboard.csv, which are the anonymized demographic and response data from the perceptual experiment, and one loose archive folder perceptual_study_videos.rar, which contains the stimuli used in the perceptual experiment. To reproduce the analysis from the paper, all files should be unzipped into the \data\raw directory of the code repository created after cloning this from GitHub. For more detail and instructions on installation, see the section of the code documentation website linked here. Usage: These recordings of live, improvised duo performances are unattributed and anonymised as agreed with participants at the point of data collection. The musicians involved received a one-off, fixed payment for their time and had their travel expenses reimbursed, with funding provided by Cambridge Digital Humanities (project page). All participants consented to the use of their recordings for projects by the current authors and for these recordings to be shared with interested members of the music psychology community, with the intention of furthering academic research. The musicians did not intend that the recordings be used for commercial, artistic, or entertainment purposes, and such use is not permitted. Citation: If you use this dataset in your research, please follow the citation format posted on GitHub. Contact: Huw Cheston - @huwcheston - [e-mail]

本数据集关联论文《网络化爵士演奏的协调策略权衡》（"Trade-offs in Coordination Strategies for Networked Jazz Performances"），包含网络化爵士二重奏即兴演奏的录音素材。该论文的预印本可在PsyArXiv平台获取。

引言：本数据集的数据采集于2022年4月至7月间的英国剑桥，源自一项博士研究项目，包含约4小时的模拟网络环境下现场即兴爵士二重奏演奏数据。数据涵盖130场独立演奏的音视频录制、生物特征数据，以及演奏者的主观评价与反馈。本项目的核心目标是通过一套新型演奏捕捉与操控系统采集数据，用于网络化音乐创作过程中合奏协调策略的实证建模。相关分析成果已发表于Cheston、Cross与Harrison（2023）的《网络化爵士演奏的协调策略权衡》（"Trade-offs in Coordination Strategies for Networked Jazz Performances"）。请参阅该文献以了解数据采集流程的完整细节。本研究的编码手册托管于GitHub平台，配合本数据集可复现论文中的分析过程。

本次录制涉及的10名演奏者，均因精通爵士即兴演奏而受邀参与本次研究。他们被划分为5组二重奏，每组各配置1名钢琴演奏者与1名鼓手，且每名演奏者仅参与一组二重奏，不重复参演。参与者被要求基于标准的12小节布鲁斯音乐结构进行即兴合奏，但需遵循特定规则：演奏出清晰明确的连续四分音符脉冲。每场演奏均会模拟不同程度的网络延迟与抖动：延迟指演奏者接收搭档实时反馈时的最小延迟时长，抖动则指该延迟的波动幅度。每场演奏所使用的延迟与抖动参数，会在对应文件或目录名称中体现，详细说明请参阅前述文献。需注意，每名二重奏组所遇到的延迟与抖动条件均为随机排布。

采集到的数据包括：
1. 每场演奏的直录线路音频（MIDI、WAV格式），含延迟与无延迟版本；
2. 由高清网络摄像头采集的视频录制文件（MKV、AVI格式）；
3. 每场演奏中每名演奏者输出的四分音符脉冲流（MIDI格式）；
4. 两名演奏者的合成音视频录制文件（MP4格式）；
5. 佩戴于手臂的设备采集的加速度计与光电容积描记（photoplethysmography，PPG）数据流（仅适用于第3至5组二重奏，存储格式为TXT）；
6. 演奏者针对每种演奏条件填写的问卷反馈（XLSX格式）；
7. 由无偏听众样本在在线感知研究中给出的演奏质量评分（CSV格式）。

仓库结构：
注：请参阅代码文档网站的对应章节，获取复现论文中分析与模型的完整指南。
文件data.zip与data.z0*分卷压缩包包含本研究采集的全部数据，但不包含感知研究的刺激素材与实验结果。解压时，请下载data.zip文件与所有后缀为.z0的对应分卷文件，使用WinRAR等支持分卷压缩的工具打开data.zip即可。请勿直接打开.z0后缀的文件，否则可能会收到数据损坏的提示。
解压data.zip后，将看到以下文件夹与文件：
- avmanip_output：每场演奏的原始MIDI、音频与视频输出。子文件夹按参与者二重奏组、实验区块与演奏条件进行组织。例如avmanip_output rial_1Block 1Condition 1 - 23 05对应首个实验会话中第一组参与者的首场演奏，为其遇到的首个条件：延迟23ms、抖动系数0.5倍。
- midi_bpm_cleaning：经过清理的MIDI文件（包含四分音符 onset 位置）。子文件夹的组织规则与avmanip_output文件夹一致。
- muxed_performances：每场演奏的合成音视频MP4文件。这些文件的命名格式为：duo_session_latency_jitter_keysfmt_drumsfmt。例如muxed_performanceskdelay_ddelayd1_s1_l23_j00_kdelay_ddelay.mp4对应首个实验会话中第一组参与者的首场演奏，延迟23ms、抖动系数0.5倍，且钢琴与鼓手端均施加了延迟与抖动。如需了解合成此类视频的更多细节，请参阅代码文档网站的对应章节。
- questionnaire_anonymized：参与者的匿名问卷反馈，相关内容也包含在关联论文的补充材料中（详见预印本）。

除data.zip与data.z0*分卷压缩包外，另有两个零散文件：Database View Participant - Dashboard.csv与Database View SuccessTrial - Dashboard.csv，分别为感知实验的匿名人口统计数据与应答数据；还有一个零散归档文件夹perceptual_study_videos.rar，内含感知研究使用的刺激素材。
如需复现论文中的分析过程，请将所有文件解压至从GitHub克隆代码仓库后创建的data
aw目录下。如需了解更多安装细节与操作指南，请参阅此处链接的代码文档网站对应章节。

使用说明：本数据集包含的现场即兴二重奏演奏录制素材均已匿名且不标注来源，这是在数据采集时与参与者达成的共识。参与的演奏者可获得一次性固定报酬，并报销差旅费用，项目资金由剑桥数字人文（Cambridge Digital Humanities，项目页面）提供。所有参与者均同意将其录制素材用于当前作者的研究项目，并同意将这些素材分享给音乐心理学领域的相关从业者，以推动学术研究发展。演奏者并未授权将录制素材用于商业、艺术或娱乐用途，此类使用行为均不被允许。

引用说明：若您在研究中使用本数据集，请遵循GitHub平台上发布的引用格式。

联系方式：Huw Cheston - @huwcheston - [e-mail]