A fMRI dataset in response to large number of short natural dynamic facial expression videos

#A fMRI dataset in response to large number of short natural dynamic facial expression videosNatural facial expressions dataset (NFED),a dataset of functional magnetic resonance imaging (fMRI) responses to 1,320 short (3s) facial expression video clips.NFED offers researchers fMRI data that enables them to investigate the neural mechanisms involved in processing emotional information communicated by facial expression videos in real-world environments.The dataset contains raw data, pre-processed volume data,pre-processed surface data and suface-based analyzed data.To get more details, please refer to the paper at {website} and the dataset at https://openneuro.org/datasets/ds005047## Preprocess procedureThe MRI data were preprocessed by using Kay et al, combining code written in MATLAB and certain tools from FreeSurfer, SPM,and FSL(http://github.com/kendrickkay). We used FreeSurfer software (http://surfer.nmr.mgh.harvard.edu) to construct the pial and white surfaces of participants from the T1 volume. Additionally, we established an intermediate gray matter surface between the pial and the white surfaces for all participants.**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/volume_pre-process/**Detailed usage notes are available in codes, please read carefully and modify variables to satisfy your customed environment.## GLM of main experimentWe utilized a single-trial General Linear Model (GLMsingle) (https://github.com/cvnlab/GLMsingle) approach, an advanced denoising approach in MATLAB R2019a, to model the pre-processed fMRI data from main experiment. For single trials, the method of GLM was developed to offer estimations of BOLD response magnitudes ('betas'). GLMsingle requires only fMRI time series data and a design matrix as inputs, integrating three techniques to enhance the accuracy of experimental GLM beta estimates. Firstly, for each voxel, a custom HRF is identified from a library of candidate functions. Secondly, cross-validation is utilized to derive a set of noise regressors from voxels unrelated to the experimental paradigm. Thirdly, to improve the stability of beta estimates for closely spaced trials, ridge regression is employed on a voxel-wise basis to regularize the betas. In this study, three betas were calculated by analyzing the BOLD response corresponding to individual video onset ranging from 1 to 3 seconds with 1-second intervals. We produced individual GLMsingle models for each session (consisted of 4 training runs and 2 test runs). In general, for each video within the training set, 2 (repetitions) x 3 (seconds) betas were acquired. Similarly, for each video within the testing set, 10 (repetitions) x 3 (seconds) betas were acquired. The utilization of repetitions enabled us to acquire video-evoked responses with a high signal-to-noise ratio (SNR).The regressors in the GLMsingle toolbox mainly include the following categories:1.Experimental Design Matrix: This is constructed based on the experimental design, including experimental conditions, task events, etc., and is used to estimate the BOLD (Blood - Oxygen - Level - Dependent) signal response.2.Data-driven nuisance regressors: The data-driven nuisance regressors used in the GLMdenoise technique. These are identified by analyzing the data itself and are used to remove noise and improve the accuracy of beta estimation.3.Physiological noise regressors: May include indicators of physiological signals such as heart rate and respiration, and are used to correct the influence of physiological noise on the BOLD signal.4.Movement parameter regressors: Usually include head movement parameters, such as translation and rotation, and are used to correct signal changes caused by head movement.5.Polynomial regressors: Polynomial terms used to o characterize the baseline signal level.**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/GLMsingle-main-experiment/matlab/NFED_GLMsingle.m**#### retinotopic mappingThe fMRI data from the the population receptive field experiment were analyzed by a pRF model implemented in the analyzePRF toolbox (http://cvnlab.net/analyzePRF/) to characterize individual retinotopic representation. Make sure to download required software mentioned in the code.**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/Functional-localizer-experiment-analysis/s4a_analysis_prf.m**#### fLoc experimentWe used GLMdenoise,a data-driven denoising method,to analyze the pre-processed fMRI data from the fLoc experiment.We used a "condition-split" strategy to code the 10 stimulus categories, splitting the trials related to each category into individual conditions in each run. Six response estimates (beta values) for each category were produced by using six condition-splits.To quantify selectivity for various categories and domains,we computed t-values using the GLM beta values after fitting the GLM.The regions of interest with category selectivity for each participant were defined by using the resulting maps.**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/Functional-localizer-experiment-analysis/s4a_analysis_floc.m**## Validation### Basic quality controlThe fundamental quality control suggests that the data displays good quality. To evaluate the quality of the structural data obtained from NFED, we employed four crucial metrics:Coefficient of Joint Variation(CJV), Contrast-to-Noise Ratio (CNR), Signal-to-Noise Ratio in Grey Matter (SNR_GM), and Signal-to-Noise Ratio in White Matter (SNR_WM). Specifically, the CJV is between white matter(WM) and grey matter(GM).The CNR assesses the relationship between the contrast of GM and WM with the noise present in the image. The SNR assesses the connection between the mean signal measurements and the noise present in the image. The SNR assessment is conducted individually for GM and WM.For quality control of the NFED’s functional scans, we assessed the amount of head motion for each participant and the temporal signal-to-noise ratio (tSNR) of the time-series data, separately.**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/T1_Image-quality-metrics/T1data/noiseeval/cal_SNRindex.m****code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/FD/FD.py****code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/tSNR/tSNR.py**### The visual cortex exhibits reliable BOLD responses to natural facial expression videos stimuli in main experiment.In the main experiment, there were 5 participants who accomplished 10 sessions, each consisting of of 4 training runs and 2 test runs, making a total of 60 runs(40 training and 20 test runs) in all. For each run, 30 (videos)*2 (repetitions) x 3 (seconds) betas were acquired. Z-score normalization was performed on the raw betas at each voxel for each run.The betas were then averaged across stimulus repetitions to generate a vector of betas. In general, 44 runs (40 training runs and 4 test runs)*90 betas were acquired. Hence, the test-retest reliability of responses to these videos in main experiment were evaluated through computing the Pearson correlation between the 90 betas obtained from the even runs and odd runs on each vertex.**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/reliability_face/reliability_face.py**### noise cellingThe code are available at https://openneuro.org/datasets/ds005047/validation/code/noise_celling/sub-xx" store the intermediate files required for running the program**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/noise_celling/Noise_Ceiling.py**### Correspondence between human brain and DCNNThe code are available at https://openneuro.org/datasets/ds005047. We combined the data from main experiment and functional localizer experiments to build an encoding model to replicate the hierarchical correspondences of representation between the brain and the DCNN. The encoding models were built to map artificial representations from each layer of the pre-trained VideoMAEv2 to neural representations from each area of the human visual cortex as defined in the multimodal parcellation atlas.**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/dnnbrain/**### Semantic metadata of action and expression labels reveal that NFED can encode temporal and spatial stimuli features in the brainThe code are available at https://openneuro.org/datasets/ds005047/validation/code/semantic_metadata/xx_xx_semantic_metadata" store the intermediate files required for running the program.**code:https://openneuro.org/datasets/ds005047/derivatives/validation/code/semantic_metadata/**## results The results can be viewed at "https://openneuro.org/datasets/ds005047/derivatives/validation/results/brain_map_individual".## Whole-brain mappingThe whole-brain data mapped to the cerebral cortex as obtained from the technical validation.**code: https://openneuro.org/datasets/ds005047/derivatives/validation/results/show_results_allbrain/Showresults.m**## Mannually prepared environmentWe provide the *requirements.txt* to install python packages used in these codes. However, some packages like *GLM* and *pre-processing* require external dependecies and we have provided the packages in the corresponding file.## stimuliThe video stimuli used in the NFED experiment are saved in the "stimuli" folders.# 面向海量短时长自然动态面部表情视频的功能磁共振成像数据集 自然面部表情数据集（Natural Facial Expressions Dataset, NFED）是一个包含1320段3秒时长面部表情视频片段的功能磁共振成像（functional magnetic resonance imaging, fMRI）响应数据集。NFED可为研究人员探究真实场景下面部表情视频传递的情绪信息加工的神经机制提供fMRI数据支持。本数据集包含原始数据、预处理体素数据、预处理表面数据以及基于表面的分析数据。如需获取更多细节，请参阅{website}处的论文以及数据集页面：https://openneuro.org/datasets/ds005047 ## 预处理流程 本数据集的MRI数据采用Kay等人的预处理流程完成，结合了MATLAB编写的代码以及FreeSurfer、SPM、FSL的相关工具（http://github.com/kendrickkay）。我们使用FreeSurfer软件（http://surfer.nmr.mgh.harvard.edu）从参与者的T1加权像体素数据中构建软脑膜表面与白质表面；此外，我们为所有参与者在软脑膜表面与白质表面之间构建了中间灰质表面。 **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/volume_pre-process/** 详细的使用说明已包含在代码中，请仔细阅读并修改变量以适配您的自定义运行环境。 ## 主实验的一般线性模型分析 我们采用单试次一般线性模型（single-trial General Linear Model, GLMsingle）方法——一款在MATLAB R2019a中实现的先进去噪方法——对主实验的预处理fMRI数据进行建模。针对单试次数据，该GLM方法可用于估计血氧水平依赖（Blood-Oxygen-Level-Dependent, BOLD）响应幅度（即“beta值”）。GLMsingle仅需fMRI时间序列数据与设计矩阵作为输入，整合了三项技术以提升实验GLM beta值估计的准确性：其一，针对每个体素，从候选函数库中识别定制化血氧响应函数（hemodynamic response function, HRF）；其二，利用交叉验证从与实验范式无关的体素中提取噪声回归因子；其三，为提升紧密间隔试次的beta值估计稳定性，我们采用逐体素岭回归对beta值进行正则化处理。 在本研究中，我们通过分析对应于1至3秒（间隔1秒）的单段视频刺激起始时刻的BOLD响应，计算得到3组beta值。我们为每个扫描会话（包含4个训练扫描段与2个测试扫描段）构建独立的GLMsingle模型。总体而言，训练集内的每段视频可获得2次重复×3秒时长对应的beta值；测试集内的每段视频可获得10次重复×3秒时长对应的beta值。通过重复采集，我们可获得高信噪比（signal-to-noise ratio, SNR）的视频诱发响应。 GLMsingle工具箱中的回归因子主要包含以下类别： 1. **实验设计矩阵**：基于实验设计构建，包含实验条件、任务事件等信息，用于估计BOLD信号响应。 2. **数据驱动干扰回归因子**：采用GLMdenoise技术中的数据驱动干扰回归因子，通过分析数据本身识别得到，用于去除噪声并提升beta值估计准确性。 3. **生理噪声回归因子**：可包含心率、呼吸等生理信号指标，用于校正生理噪声对BOLD信号的影响。 4. **头动参数回归因子**：通常包含平移与旋转等头部运动参数，用于校正由头部运动导致的信号变化。 5. **多项式回归因子**：用于表征基线信号水平的多项式项。 **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/GLMsingle-main-experiment/matlab/NFED_GLMsingle.m** ### 视网膜拓扑映射 我们采用analyzePRF工具箱（http://cvnlab.net/analyzePRF/）中实现的群体感受野（population receptive field, pRF）模型，对群体感受野实验的fMRI数据进行分析，以表征个体的视网膜拓扑表征。请确保下载代码中提及的所需软件。 **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/Functional-localizer-experiment-analysis/s4a_analysis_prf.m** ### fLoc实验 我们采用数据驱动去噪方法GLMdenoise对fLoc实验的预处理fMRI数据进行分析。我们使用“条件拆分”策略对10类刺激类别进行编码，将每类刺激相关的试次在每个扫描段中拆分为独立条件。通过6次条件拆分，我们为每个类别计算得到6组响应估计值（beta值）。为量化不同类别与脑区的选择性，我们在拟合GLM后利用GLM beta值计算t值。我们通过得到的激活图为每位参与者定义了具有类别选择性的感兴趣区（regions of interest, ROI）。 **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/Functional-localizer-experiment-analysis/s4a_analysis_floc.m** ## 验证 ### 基础质量控制 本数据集的基础质量控制结果表明数据质量良好。为评估NFED结构数据的质量，我们采用四项关键指标：联合变异系数（Coefficient of Joint Variation, CJV）、对比度噪声比（Contrast-to-Noise Ratio, CNR）、灰质信噪比（Signal-to-Noise Ratio in Grey Matter, SNR_GM）以及白质信噪比（Signal-to-Noise Ratio in White Matter, SNR_WM）。其中，CJV用于表征白质与灰质之间的变异关系；CNR用于评估灰质与白质的对比度与图像噪声之间的关系；SNR分别针对灰质与白质，评估平均信号测量值与图像噪声之间的关系。针对NFED功能扫描的质量控制，我们分别评估了每位参与者的头部运动情况以及时间序列数据的时间信噪比（temporal signal-to-noise ratio, tSNR）。 **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/T1_Image-quality-metrics/T1data/noiseeval/cal_SNRindex.m** **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/FD/FD.py** **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/tSNR/tSNR.py** ### 主实验中视觉皮层对自然面部表情视频刺激具有可靠的BOLD响应 主实验共有5名参与者完成了10个扫描会话，每个会话包含4个训练扫描段与2个测试扫描段，总计60个扫描段（40个训练段与20个测试段）。每个扫描段可获得30段视频×2次重复×3秒时长对应的beta值。我们对每个扫描段中每个体素的原始beta值进行Z分数归一化，随后对刺激重复次数对应的beta值取平均，得到beta值向量。总体而言，我们共获得44个扫描段（40个训练段与4个测试段）×90组beta值。因此，我们通过计算每位皮层顶点在奇偶扫描段中得到的90组beta值之间的皮尔逊相关系数，评估了主实验中视频响应的重测信度。 **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/reliability_face/reliability_face.py** ### 噪声上限 相关代码可于https://openneuro.org/datasets/ds005047/validation/code/noise_celling/sub-xx 获取，该路径用于存储程序运行所需的中间文件。 **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/noise_celling/Noise_Ceiling.py** ### 人脑与深度卷积神经网络的对应关系 相关代码可于https://openneuro.org/datasets/ds005047 获取。我们结合主实验与功能定位实验的数据构建编码模型，以复现人脑与深度卷积神经网络（Deep Convolutional Neural Network, DCNN）之间的分层表征对应关系。我们构建的编码模型用于将预训练VideoMAEv2各层的人工表征映射至人类视觉皮层各脑区的神经表征，映射依据多模态分区图谱定义的脑区边界。 **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/dnnbrain/** ### 动作与表情标签的语义元数据表明NFED可编码大脑中的时空刺激特征 相关代码可于https://openneuro.org/datasets/ds005047/validation/code/semantic_metadata/xx_xx_semantic_metadata 获取，该路径用于存储程序运行所需的中间文件。 **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/code/semantic_metadata/** ## 结果 结果可通过以下链接查看：https://openneuro.org/datasets/ds005047/derivatives/validation/results/brain_map_individual ## 全脑映射 将技术验证得到的全脑数据映射至大脑皮层。 **代码链接：https://openneuro.org/datasets/ds005047/derivatives/validation/results/show_results_allbrain/Showresults.m** ## 手动配置运行环境 我们提供了requirements.txt文件用于安装本代码所需的Python包。不过部分模块（如GLM与预处理相关工具）需要外部依赖项，我们已在对应文件中提供了所需的安装包。 ## 刺激材料 NFED实验中使用的视频刺激材料存储于"stimuli"文件夹中。