Physiological Signals During Motor Imagery Brain-Computer Interface Training Using Virtual Reality and Haptics

Participant demographics:

The sample is consisted by 20 healthy volunteers with a mean age of 24.79 years (SD = 3.54 years).  The cohort was 68% male and 32% female.  In terms of education, 16% had attended only high school, while 32% had a bachelor's degree, 42% a master's degree, and 11% a doctorate. All participants signed an informed consent before participating in the study in accordance with the 1964 Declaration of Helsinki.

Experiment Description:

The experiment consisted in having the subjects perform motor imagery of a bimanual rowing task with two individual paddles, one in each hand, under five experimental conditions. Four of these conditions used NeuRow (Vourvopoulos et al. (2016-2019))—a VR environment that renders virtual arms from a first-person perspective—while the other conditions used abstract feedback based on the BCI-Graz paradigm (Pfurtscheller et al. (2003)). All six conditions and their acronyms are described below:

Motor Imagery(MI): The standard motor imagery training, with a fixation cross and directional arrows on a black background guiding the subjects through the experiment.

Motor Imagery/Motor Observation (MIMO): A motor imagery training paradigm using NeuRow, with a fixation cross and directional arrows overlaid on the VR environment, which was displayed through a monitor.

Motor Imagery/Motor Observation with Haptics (MIMOHP): A motor imagery training paradigm using NeuRow, with a fixation cross and directional arrows overlaid on the VR environment, which was displayed through a monitor. Hand controllers also provided haptic feedback through vibrotactile stimulation.

Motor Imagery/Motor Observation with VR HMD (MIMOVR): A motor imagery training paradigm using NeuRow, with a fixation cross and directional arrows overlaid on the VR environment, which was displayed through a VR HMD.

Motor Imagery/Motor Observation with VR HMD and Haptics (MIMOVRHP): A motor imagery training paradigm using NeuRow, with a fixation cross and directional arrows overlaid on the VR environment, which was displayed through a VR HMD. Hand controllers also provided haptic feedback through vibrotactile stimulation.

Motor Execution (ME): A fixation cross and directional arrows were displayed on a black background through a monitor (same as in MI), and guided the subjects through the experiment by having them tap their fingers accordingly. Data from this condition was available only after S07, so only 10 subjects
have performed ME.

Finally, this experiment followed a within-subject design, in a randomized order of the conditions to minimize any order effects, while MI and ME conditions acted as control.

Equipment:

A wireless EEG amplifier (LiveAmp; Brain Products GmbH, Gilching, Germany) was used, with 32 active electrodes(+3 ACC) with a sampling rate of 500Hz. In addition, ECG, PPG and Respiration signals have been recorded synchronously in a bipolar montage, and connected to the EEG amplifier’s AUX input through the Brain Products BIP2AUX adapter.

Visual feedback was provided through a monitor in all conditions except in MIMOVR and MIMOVRHP, in which an Oculus Rift CV1 headset (Reality Labs, formerly Facebook, Inc., CA, USA) was used instead. Haptic feedback was provided through the Oculus Rift hand controllers.

Channel Indices:

EEG: 1-32
PPG (AUX1): 33
Resp. (AUX2): 34
ECG (AUX3): 35
ACC: 36-38

Event codes:

Code
Description

S01
Experiment Start

S02
Baseline Start

S03
Baseline Stop

S04
Start Of Trial

S05
Cross On Screen

S07
class1, Left hand 

S08
class2, Right hand 

S09
Feedback Continuous

S10
End of Trial

S11
End Of Session

S12
Experiment Stop

Directory tree:

ROOT
|
+--- USER #
|            +---SESSION #
|            |            +---TASK #
|            |            |             +---MI
|            |            |             |           .eeg
|            |            |             |           .vhdr
|            |            |             |           .vmrk
|            |            |            +---MIMO
|            |            |             |           .eeg
|            |            |             |           .vhdr
|            |            |             |           .vmrk
|            |            |            +---MIMOHP
|            |            |             |           .eeg
|            |            |             |           .vhdr
|            |            |             |           .vmrk
|            |            |            +---MIMOVR
|            |            |             |           .eeg
|            |            |             |           .vhdr
|            |            |             |           .vmrk
|            |            |            +---MIMOHPVR
|            |            |             |           .eeg
|            |            |             |           .vhdr
|            |            |             |           .vmrk
|            |            |             +---ME
|            |            |             |           .eeg
|            |            |             |           .vhdr
|            |            |             |           .vmrk

Note: The first three datasets are from pilot sessions: sub-p01 to p03. From sub-01 to 19, subjects 10 and 11 have been removed due to the lack of markers. Subject sub-13, task MIMOVRHP is missing.