Stabilizing Perturbation Propagation in Multimodal Emotion Recognition via Oscillatory Mamba Dynamics

Multimodal emotion recognition (MER) integrates various signals through cross-modal alignment and fusion. While earlier studies often blamed system fragility on fusion-level interactions, our analysis shows that input-level perturbations cause more immediate degradation. However, this impact depends on how perturbations travel across representation, fusion, and decision layers. We find that the fusion pathway can amplify or reduce perturbations, rather than being solely responsible for vulnerability. To manage this propagation, we propose the Dual-Oscillatory Mamba Fusion Network, which uses dynamic contraction and synchronization to stabilize decision-state trajectories. Extensive experiments prove that our framework effectively reduces both input-level and fusion-level adversarial perturbations, achieving improved robustness and maintaining accuracy and macro-F1. This work shifts focus from vulnerability to perturbation propagation, providing a new approach for robust multimodal learning. Code.rar contains the source code of the proposed method. The file affectnet-8.rar includes heatmaps of both original and adversarial attacks corresponding to input-level images and pathways. The remaining files provide a diverse set of adversarial attack heatmaps that illustrate fusion-level interactions in Multimodal Emotion Recognition. These results are utilized to validate and demonstrate the effectiveness of the proposed Oscillatory Mamba Dynamics framework. Guoming Chen and Yuting Liao contributed equally to this work and they are co–first authors.多模态情感识别 (Multimodal Emotion Recognition, MER) 通过跨模态对齐与融合整合多类信号。早期研究常将系统脆弱性归咎于融合层级的交互作用，但本研究分析表明，输入层级的扰动会引发更直接的性能退化。然而，该影响取决于扰动在表征层、融合层与决策层间的传播路径。我们发现，融合通路既可以放大也可以削弱扰动，而非仅为系统脆弱性的唯一诱因。为管控此类扰动传播，我们提出双振荡Mamba融合网络 (Dual-Oscillatory Mamba Fusion Network)，通过动态收缩与同步机制稳定决策状态轨迹。大量实验证明，所提框架可有效降低输入层级与融合层级的对抗扰动，在提升鲁棒性的同时维持了模型准确率与宏F1值。本研究将关注点从脆弱性转向扰动传播，为鲁棒多模态学习提供了全新的解决方案。 Code.rar 包含所提方法的源代码。文件 affectnet-8.rar 收录了与输入层级图像及通路对应的原始攻击与对抗攻击热力图。其余文件则提供了多样化的对抗攻击热力图，用于展示多模态情感识别中的融合层级交互作用。上述结果可用于验证并证明所提振荡Mamba动力学框架的有效性。 陈果明与廖雨婷对本工作贡献均等，为共同第一作者。