Bimodal dataset on Inner speech

Bimodal dataset on Inner Speech

Code available: https://github.com/LTU-Machine-Learning/Inner_Speech_EEG_FMRI

Publication available: https://www.nature.com/articles/s41597-023-02286-w

Abstract:
The recognition of inner speech, which could give a `voice' to patients that have no ability to speak or move, is a challenge for brain-computer interfaces (BCIs). A shortcoming of the available datasets is that they do not combine modalities to increase the performance of inner speech recognition. Multimodal datasets of brain data enable the fusion of neuroimaging modalities with complimentary properties, such as the high spatial resolution of functional magnetic resonance imaging (fMRI) and the temporal resolution of electroencephalography (EEG), and therefore are promising for decoding inner speech. This paper presents the first publicly available bimodal dataset containing EEG and fMRI data acquired nonsimultaneously during inner-speech production. Data were obtained from four healthy, right-handed participants during an inner-speech task with words in either a social or numerical category. Each of the 8-word stimuli were assessed with 40 trials, resulting in 320 trials in each modality for each participant.
The aim of this work is to provide a publicly available bimodal dataset on inner speech, contributing towards speech prostheses.

Short Dataset description:
The dataset consists of 1280 trials in each modality (EEG, FMRI).
The stimuli contain 8 words, selected from 2 different categories (social, numeric):
Social: child, daughter, father, wife
Numeric: four, three, ten, six

There are 4 subjects in total: sub-01, sub-02, sub-03, sub-05. Initially, there were 5 participants, however, sub-04 data was rejected due to high fluctuations. Details of valid data are available in the file participants.tsv.

Note that the EEG dataset shared on OpenNeuro includes recordings that have been preprocessed to remove MRI gradient artefacts and cardioballistic artefacts using average artefact subtraction (AAS). R-peak events were extracted from the ECG channel after gradient artefact correction and are included in the .vmrk files. Raw, unprocessed EEG data are not included in the public repository but can be made available upon reasonable request.

For questions please contact: [e-mail]