Multimodal datataset of EEG, ECG and audio recordings of individual musicians playing emotional music

This dataset contains EEG, ECG and audio recordings of 11 individual musicians playing emotional music on their instrument. The dataset consists of two parts:

Experiment: Musicians’ self-reported ratings, audio recordings, and physiological recordings where the 11 expert musicians were asked to play at least 4~2-minute unfamiliar (non-popularly known) musical pieces. Participants were asked to play at least once one of the following  emotions:  happiness, sadness, relaxation, and anger. For the  physiological recordings EEG, ECG, and GSR signals were recorded. Each musican’s data is denoted by MS_ followed by the order in which they were recorded.

Self-report questionnaire: A self-assessment questionnaire and their answers where 11 expert musicians were asked to rate musical pieces recorded based on:

Objective valence & arousal they felt the piece had;

Felt valence & arousal during playing.

For a more detailed explanation of the dataset, its recording procedure, and its contents, see 

L. Turchet, B. O'Sullivan, R. Ortner & C. Gugher (2024). Emotion Recognition of Playing Musicians from EEG, ECG, and Acoustic Signals. IEEE Transactions on Human-Machine Systems.

File Listing

The following files are available (each explained in more detail below):

Name

Format

Contents

EEG_ECG_data_for_each_musician

mat

This folder contains the raw EEG, ECG, & GSR data for all 11 musicians for each piece they played as well as a resting state recording, which was recorded while a neutral audio stimulus was played.

audio_data_for_each_musician

wav, JSON, csv

This folder contains 3 subfolders: 1) wav_audio_files_original: This is the raw audio data recorded for each musician and includes all 56 pieces included in the paper reported above, please see below regarding rejected trials.

2) wav_audio_files (original split_into_3_parts): Here, the 56 pieces are appropriately split into 3 separate parts.3) analysis_audio_files: This folder contains the acoustic features extracted, 1714 acoustic features were extracted from each split trial. Each result is stored in .JSON, however, the collated results can be seen in all_results.csv.

self_report_questionnaire

pdf, xls

Two files exist in this folder:1. The self-reported questionnaire given to the musicians of the questionnaire during the experiment.2. 

File Details

EEG_ECG_data_for_each_musician

These are the original raw data recordings. EEG data were recorded using a g.GAMMAcap2 by g.tec Medical Engineering, a 64-channel cap with g.SCARABEO active electrodes, with two g.GAMMAsys reference active ear clip electrodes. Two g.GAMMAbox electrode connector boxes were used to connect the active electrodes to two g.USBamp biosignal amplifiers with a sampling frequency of 256 Hz.The following 31 EEG channels were used: Fp1, Fp2, AFz, AF3, AF4, AF7, AF8, Fz, F3, F4, F7, F8, Cz, C3, C4, CP3, CP4, CP5, CP6, P1, P2, P3, P4, P5, P6, P7, P8, PO7, PO8, O1, and O2. AFz was used as a ground electrode and Cz was used as a re-reference electrode. The right-side ear clip electrode was used as a reference electrode. 

ECG data were recorded using a single g.GAMMAclip active electrode clip connected directly to the g.GAMMAbox, sharing the same ground electrode with the EEG cap and placed on position V4 of the subjects.GSR data was recorded using the g.GSRsensor² box which contains two small dry electrodes placed underneath the participant’s toes (due to the amount of hand movement required for playing an instrument). The g.GSRsensor² was connected directly to a g.GAMMAbox, using jumper cables connected to different ports in the g.USBAMPs to share the same reference and ground electrodes as the EEG cap. 

The locations of the channels and their corresponding number in the raw data is as follows:

Channel Number

Channel Name

1

Time series

2

AF3

3

AF4

4

AF7

5

AF8

6

CP3

7

CP4

8

CP5

9

CP6

10

P1

11

P2

12

P5

13

P6

14

P7

15

P8

16

O1

17

O2

18

Cz

19

Fp1

20

Fp2

21

F3

22

F4

23

F7

24

F8

25

C3

26

C4

27

P3

28

P4

29

PO7

30

PO8

31

AFz

32

GSR

33

ECG

audio_data_for_each_musician/wav_audio_files_original

This folder contains all of the raw audio files recorded during the experiment. All pieces were recorded using the software Audacity and exported as WAV files encoded with a bit depth of 32-bits and a sampling rate of 44.1 kHz. 56 of the included trials are present in this folder.

audio_data_for_each_musician/wav_audio_files (original split_into_3_parts)

This folder contains the above mentioned 56 raw audio pieces separated into 3 appropriately sized recordings.

audio_data_for_each_musician/analysis_audio_filesThis folder contains the 1714 acoustic features from each of the 3 separated trials from 56 accepted pieces, these are denoted by MS_ followed by the order in which the musicians were recorded and the order in which the trails were split, the individual features are in JSON format. Within this folder, we have collated all results in a csv file (all_results.csv) where the columns show the intended emotion, trial name and number, followed by the names of the acoustic features. 

self_report_questionnaire

This pdf is the questionnaire which each musician was given following each trial relating to the emotions communicated and felt during each trial. Most* questions in the questionnaire were multiple-choice and speak pretty much for themselves. The answers for which were collated and are described below.

self_report_questionnaire_anwsers

This .xls file contains the results of all the musicians self-reported ratings from the above described questionnaire. 

Column name

Description

Subject

The subject code of the musician, denoted by MS_

recording_filename

The name of the trial denoted by MS_01_ followed by the trial number.

intended_emotion

The intended emotion the musician was instructed to communicate. The emotions are as follows:

Angry

Sad

Relaxed

Happy

They are intended to be reported by using the valence-arousal space.

valence_communicated

The valence rating (integer between 1 and 5), that participants were asked to objectively rate how they thought the music played would be perceived. 

arousal_communicated

As above, but relating to arousal rather than valence.

valence_felt

The valence rating (integer between 1 and 5), that participants were asked to rate how they felt while playing the piece.

arousal_felt

As above, but relating to arousal rather than valence.

instrument_played

The instrument played for each trial.