Simultaneous EEG and fNIRS recordings for semantic decoding of imagined animals and tools

### Description
This dataset contains simultaneous electroencephalography (EEG) and near-infrared spectroscopy (fNIRS) signals recorded from 12 participants while performing a silent naming task and three sensory-based imagery tasks using visual, auditory, and tactile perception.
Participants were asked to visualize an object in their minds, imagine the sounds made by the object, and imagine the feeling of touching the object.

### EEG
EEG data were acquired with a BioSemi ActiveTwo system with 64 electrodes positioned according to the international 10-20 system, plus one electrode on each earlobe as references ('EXG1' channel is the left ear electrode and 'EXG2' channel is the right ear electrode).
Additionally, 2 electrodes placed on the left hand measured galvanic skin response ('GSR1' channel) and a respiration belt around the waist measured respiration ('Resp' channel).
The sampling rate was 2048 Hz.

The electrode names were saved in a default BioSemi labeling scheme (A1-A32, B1-B32). See the Biosemi documentation for the corresponding international 10-20 naming scheme (https://www.biosemi.com/pics/cap_64_layout_medium.jpg, https://www.biosemi.com/headcap.htm).

For convenience, the following ordered channels
```
['A1', 'A2', 'A3', 'A4', 'A5', 'A6', 'A7', 'A8', 'A9', 'A10', 'A11', 'A12', 'A13', 'A14', 'A15', 'A16', 'A17', 'A18', 'A19', 'A20', 'A21', 'A22', 'A23', 'A24', 'A25', 'A26', 'A27', 'A28', 'A29', 'A30', 'A31', 'A32', 'B1', 'B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'B8', 'B9', 'B10', 'B11', 'B12', 'B13', 'B14', 'B15', 'B16', 'B17', 'B18', 'B19', 'B20', 'B21', 'B22', 'B23', 'B24', 'B25', 'B26', 'B27', 'B28', 'B29', 'B30', 'B31', 'B32']
```
can thus be renamed to
```
['Fp1', 'AF7', 'AF3', 'F1', 'F3', 'F5', 'F7', 'FT7', 'FC5', 'FC3', 'FC1', 'C1', 'C3', 'C5', 'T7', 'TP7', 'CP5', 'CP3', 'CP1', 'P1', 'P3', 'P5', 'P7', 'P9', 'PO7', 'PO3', 'O1', 'Iz', 'Oz', 'POz', 'Pz', 'CPz', 'Fpz', 'Fp2', 'AF8', 'AF4', 'AFz', 'Fz', 'F2', 'F4', 'F6', 'F8', 'FT8', 'FC6', 'FC4', 'FC2', 'FCz', 'Cz', 'C2', 'C4', 'C6', 'T8', 'TP8', 'CP6', 'CP4', 'CP2', 'P2', 'P4', 'P6', 'P8', 'P10', 'PO8', 'PO4', 'O2']
```

### fNIRS
fNIRS data were acquired with a NIRx NIRScoutXP continuous wave imaging system equipped with 4 light detectors, 8 light emitters (sources), and low-profile fNIRS optodes.
Both electrodes and optodes were placed in a NIRx NIRScap for integrated fNIRS-EEG layouts.
Two different montages were used: frontal and temporal, see references for more information.

### Stimulus
Folder 'stimuli' contains all images of the semantic categories of animals and tools presented to participants.

### Example code
We have prepared example scripts to demonstrate how to load the EEG and fNIRS data into Python using MNE and MNE-BIDS packages. These scripts are located in the 'code' directory.

### References
This dataset was analyzed in the following publications:

[1] Rybář, M., Poli, R. and Daly, I., 2024. Using data from cue presentations results in grossly overestimating semantic BCI performance. Scientific Reports, 14(1), p.28003.

[2] Rybář, M., Poli, R. and Daly, I., 2021. Decoding of semantic categories of imagined concepts of animals and tools in fNIRS. Journal of Neural Engineering, 18(4), p.046035.

[3] Rybář, M., 2023. Towards EEG/fNIRS-based semantic brain-computer interfacing (Doctoral dissertation, University of Essex).

### 数据集概述
本数据集收录了12名被试在完成静默命名任务，以及视觉、听觉、触觉三类感官想象任务时同步采集的脑电图（electroencephalography, EEG）与功能性近红外光谱（near-infrared spectroscopy, fNIRS）信号。实验要求被试在脑海中具象化目标物体、想象该物体发出的声响，以及模拟触摸该物体的触感。

### EEG数据
本数据集的EEG数据采用BioSemi ActiveTwo系统采集，共配置64个电极，依照国际10-20系统布局，且在双耳耳垂各设置1个参考电极（'EXG1'通道对应左耳电极，'EXG2'通道对应右耳电极）。
此外，左手佩戴的2个电极用于采集皮肤电反应（'GSR1'通道），腰部佩戴的呼吸带用于采集呼吸信号（'Resp'通道）。采样率为2048 Hz。
电极名称采用BioSemi默认标注方案（A1-A32、B1-B32）。如需查询对应国际10-20命名规范，可参考BioSemi官方文档：https://www.biosemi.com/pics/cap_64_layout_medium.jpg、https://www.biosemi.com/headcap.htm。
为便于使用，以下有序通道列表可直接重命名：

['A1', 'A2', 'A3', 'A4', 'A5', 'A6', 'A7', 'A8', 'A9', 'A10', 'A11', 'A12', 'A13', 'A14', 'A15', 'A16', 'A17', 'A18', 'A19', 'A20', 'A21', 'A22', 'A23', 'A24', 'A25', 'A26', 'A27', 'A28', 'A29', 'A30', 'A31', 'A32', 'B1', 'B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'B8', 'B9', 'B10', 'B11', 'B12', 'B13', 'B14', 'B15', 'B16', 'B17', 'B18', 'B19', 'B20', 'B21', 'B22', 'B23', 'B24', 'B25', 'B26', 'B27', 'B28', 'B29', 'B30', 'B31', 'B32']

重命名后为：

['Fp1', 'AF7', 'AF3', 'F1', 'F3', 'F5', 'F7', 'FT7', 'FC5', 'FC3', 'FC1', 'C1', 'C3', 'C5', 'T7', 'TP7', 'CP5', 'CP3', 'CP1', 'P1', 'P3', 'P5', 'P7', 'P9', 'PO7', 'PO3', 'O1', 'Iz', 'Oz', 'POz', 'Pz', 'CPz', 'Fpz', 'Fp2', 'AF8', 'AF4', 'AFz', 'Fz', 'F2', 'F4', 'F6', 'F8', 'FT8', 'FC6', 'FC4', 'FC2', 'FCz', 'Cz', 'C2', 'C4', 'C6', 'T8', 'TP8', 'CP6', 'CP4', 'CP2', 'P2', 'P4', 'P6', 'P8', 'P10', 'PO8', 'PO4', 'O2']

### fNIRS数据
本数据集的fNIRS数据采用NIRx NIRScoutXP连续波成像系统采集，该系统配备4个光探测器、8个光发射器（光源）以及超薄型fNIRS光学电极。
电极与光学电极均安装于NIRx NIRScap帽中，以实现fNIRS与EEG的一体化布局。实验采用两种不同的电极布局方案：额叶区与颞叶区，详细信息可参考相关文献。

### 刺激材料
名为`stimuli`的文件夹收录了向被试呈现的动物与工具两类语义类别的全部图像。

### 示例代码
我们已编写示例脚本，演示如何通过MNE与MNE-BIDS工具包将EEG与fNIRS数据加载至Python环境中，相关脚本存放于`code`目录下。

### 参考文献
本数据集的相关分析发表于以下出版物：
[1] Rybář M, Poli R, Daly I. 2024. 利用线索呈现数据会严重高估语义脑机接口的性能[J]. 科学报告, 14(1): 28003.
[2] Rybář M, Poli R, Daly I. 2021. 基于fNIRS的动物与工具想象概念语义类别解码[J]. 神经工程期刊, 18(4): 046035.
[3] Rybář M. 2023. 面向EEG/fNIRS的语义脑机接口研究[D]. 埃塞克斯大学, 2023.

搜集汇总

背景与挑战

背景概述

该数据集包含12名参与者的同步脑电图（EEG）和功能性近红外光谱（fNIRS）记录，用于语义解码想象动物和工具。数据采集涉及无声命名任务和多种感官想象任务，使用高采样率（2048 Hz）的EEG系统和特定布局的fNIRS设备，并提供了刺激图像和示例代码以支持神经信号分析研究。