A large-scale fMRI dataset for human action recognition

**Summary**

Human action recognition is one of our critical living abilities, allowing us to interact easily with the environment and others in everyday life. Although the neural basis of action recognition has been widely studied using a few categories of actions from simple contexts as stimuli, how the human brain recognizes diverse human actions in real-world environments still need to be explored. Here, we present the Human Action Dataset (HAD), a large-scale functional magnetic resonance imaging (fMRI) dataset for human action recognition. HAD contains fMRI responses to 21,600 video clips from 30 participants. The video clips encompass 180 human action categories and offer a comprehensive coverage of complex activities in daily life. We demonstrate that the data are reliable within and across participants and, notably, capture rich representation information of the observed human actions. This extensive dataset, with its vast number of action categories and exemplars, has the potential to deepen our understanding of human action recognition in natural environments.

**Data record**

The data were organized according to the Brain-Imaging-Data-Structure (BIDS) Specification version 1.7.0 and can be accessed from the OpenNeuro public repository (accession number: ds004488). The raw data of each subject were stored in "sub-< ID>" directories. The preprocessed volume data and the derived surface-based data were stored in “derivatives/fmriprep” and “derivatives/ciftify” directories, respectively. The video clips stimuli were stored in “stimuli” directory.

*Video clips stimuli*
The video clips stimuli selected from HACS are deposited in the "stimuli" folder. Each of the 180 action categories holds a folder in which 120 unique video clips are stored.

*Raw data*
The data for each participant are distributed in three sub-folders, including the “anat” folder for the T1 MRI data, the “fmap” folder for the field map data, and the “func” folder for functional MRI data. The events file in “func” folder contains the onset, duration, trial type (category index) in specific scanning run.

*Preprocessed volume data from fMRIprep*
The preprocessed volume-based fMRI data are in subject's native space, saved as “sub-<subID>_ses-<sesID>_task-<taskID>_run-<index>_space-T1w_desc-preproc_bold.nii.gz” for each functional run. A “sub-<subID>_ses-<sesID>_task-<taakID>_run-<index>_desc-confounds_timeseries.tsv” file stores the confounds variable extracted by fMRIPrep. Other auxiliary files can also be found under each session folder.

*Preprocessed surface data from ciftify*
Under the “results” folder, the preprocessed surface-based data are saved in standard fsLR space, named as “sub-<subID>/results/ses-action01_task-action_run-<index>_Atlas.dtseries.nii” for each functional run. The standard and native fsLR surface can be found in the “standard_fsLR_surface” and “T1w_fsLR_surface” folders, respectively.
The brain activation data derived from GLM analyses are saved as “sub-<subID>/results/ses-action01_task-action_cycle-<cycleIndex>_beta.dscalar.nii” for each cycle corresponded with every three runs. The auxiliary information about labels or conditions can be found in “ses-action01_task-action_cycle-<cycleIndex>_label.txt”.

**摘要**
人类动作识别是人类至关重要的生存能力之一，可使我们在日常生活中轻松与环境及他人开展互动。尽管学界已通过少量来自简单场景的动作类别作为刺激物，对动作识别的神经基础开展了广泛研究，但人类大脑如何在真实世界环境中识别多样化的人类动作，仍有待进一步探索。本研究发布**人类动作数据集（Human Action Dataset, HAD）**，这是一个用于人类动作识别的大规模功能磁共振成像（functional magnetic resonance imaging, fMRI）数据集。HAD包含30名被试对21600个视频片段的fMRI响应数据；这些视频片段涵盖180个人类动作类别，全面覆盖了日常生活中的复杂活动。研究表明，该数据在被试内部及跨被试间均具有可靠性，尤为关键的是，其能够捕获所观察到的人类动作的丰富表征信息。这款涵盖海量动作类别与样本的大型数据集，有望加深我们对自然环境中人类动作识别机制的理解。

**数据记录**
本数据集按照《脑成像数据结构（Brain-Imaging-Data-Structure, BIDS）规范1.7.0版》进行组织，可从OpenNeuro公共仓库获取（存档编号：ds004488）。每名被试的原始数据存储于"sub-<ID>"目录中。预处理的体素数据与衍生的基于表面的数据，分别存储于"derivatives/fmriprep"与"derivatives/ciftify"目录。视频片段刺激物存储于"stimuli"目录。

*视频片段刺激物*
从HACS中选取的视频片段刺激物存放于"stimuli"文件夹内。180个动作类别各对应一个子文件夹，每个子文件夹中存储有120个独特的视频片段。

*原始数据*
每名被试的数据分布于三个子文件夹中："anat"文件夹用于存放T1磁共振成像数据，"fmap"文件夹用于存放场图数据，"func"文件夹用于存放功能磁共振成像数据。"func"文件夹中的事件文件包含了对应扫描序列中各试次的刺激呈现时刻、持续时长以及试次类型（类别索引）。

*fMRIPrep预处理体素数据*
经fMRIPrep处理的预处理体素式fMRI数据采用被试原生空间存储，每个功能扫描序列的文件命名格式为："sub-<subID>_ses-<sesID>_task-<taskID>_run-<index>_space-T1w_desc-preproc_bold.nii.gz"。文件"sub-<subID>_ses-<sesID>_task-<taskID>_run-<index>_desc-confounds_timeseries.tsv"存储了fMRIPrep提取的混淆变量。每个会话文件夹下还可获取其他辅助文件。

*ciftify预处理表面数据*
在"results"文件夹下，预处理的基于表面的数据采用标准fsLR空间存储，每个功能扫描序列的文件命名为："sub-<subID>/results/ses-action01_task-action_run-<index>_Atlas.dtseries.nii"。标准fsLR表面与原生fsLR表面分别可于"standard_fsLR_surface"与"T1w_fsLR_surface"文件夹中获取。
从一般线性模型（General Linear Model, GLM）分析得到的脑激活数据，每个对应每3个扫描序列的循环的文件命名为："sub-<subID>/results/ses-action01_task-action_cycle-<cycleIndex>_beta.dscalar.nii"。关于标签或实验条件的辅助信息可于"ses-action01_task-action_cycle-<cycleIndex>_label.txt"中查看。

搜集汇总

背景与挑战

背景概述

该数据集是一个大规模的功能磁共振成像(fMRI)数据集，专为人类动作识别设计，包含30名参与者对21,600个视频片段的fMRI响应，涵盖180个日常人类动作类别，旨在探索大脑在自然环境中识别复杂动作的神经机制。数据遵循BIDS规范，提供原始和预处理版本，具有高可靠性和丰富的表征信息，适用于神经影像学和行为研究。