Multimodal Functional Network Connectivity: An EEG-fMRI Fusion in Network Space

EEG and fMRI recordings measure the functional activity of multiple coherent networks distributed in the cerebral cortex. Identifying network interaction from the complementary neuroelectric and hemodynamic signals may help to explain the complex relationships between different brain regions. In this paper, multimodal functional network connectivity (mFNC) is proposed for the fusion of EEG and fMRI in network space. First, functional networks (FNs) are extracted using spatial independent component analysis (ICA) in each modality separately. Then the interactions among FNs in each modality are explored by Granger causality analysis (GCA). Finally, fMRI FNs are matched to EEG FNs in the spatial domain using network-based source imaging (NESOI). Investigations of both synthetic and real data demonstrate that mFNC has the potential to reveal the underlying neural networks of each modality separately and in their combination. With mFNC, comprehensive relationships among FNs might be unveiled for the deep exploration of neural activities and metabolic responses in a specific task or neurological state.

脑电图（Electroencephalogram, EEG）与功能磁共振成像（functional Magnetic Resonance Imaging, fMRI）能够采集分布于大脑皮层的多个协同网络的功能活动信号。从互补的神经电信号与血液动力学信号中识别网络交互机制，有助于阐释不同脑区间的复杂关联。本文提出了多模态功能网络连接（multimodal functional network connectivity, mFNC）方法，用于在网络空间中实现脑电图与功能磁共振成像的信号融合。首先，分别针对两种模态数据，通过空间独立成分分析（spatial independent component analysis, ICA）提取功能网络（functional networks, FNs）；随后，借助格兰杰因果分析（Granger causality analysis, GCA）探究各模态下功能网络间的交互关系；最后，通过基于网络的源成像（network-based source imaging, NESOI）方法，在空间域中完成功能磁共振成像功能网络与脑电图功能网络的匹配。针对合成数据与真实数据的实验结果均表明，mFNC能够分别且联合地揭示各模态下的潜在神经网络机制。借助mFNC方法，可进一步揭示功能网络间的全面关联，从而为特定任务或神经状态下的神经活动与代谢响应的深入研究提供支撑。