Stabilizing Perturbation Propagation in Multimodal Emotion Recognition via Oscillatory Mamba Dynamics

Multimodal emotion recognition (MER) integrates various signals through cross-modal alignment and fusion. While earlier studies often blamed system fragility on fusion-level interactions, our analysis shows that input-level perturbations cause more immediate degradation. However, this impact depends on how perturbations travel across representation, fusion, and decision layers. We find that the fusion pathway can amplify or reduce perturbations, rather than being solely responsible for vulnerability. To manage this propagation, we propose the Dual-Oscillatory Mamba Fusion Network, which uses dynamic contraction and synchronization to stabilize decision-state trajectories. Extensive experiments prove that our framework effectively reduces both input-level and fusion-level adversarial perturbations, achieving improved robustness and maintaining accuracy and macro-F1. This work shifts focus from vulnerability to perturbation propagation, providing a new approach for robust multimodal learning. Code.rar contains the source code of the proposed method. The file affectnet-8.rar includes heatmaps of both original and adversarial attacks corresponding to input-level images and pathways. The remaining files provide a diverse set of adversarial attack heatmaps that illustrate fusion-level interactions in Multimodal Emotion Recognition. These results are utilized to validate and demonstrate the effectiveness of the proposed Oscillatory Mamba Dynamics framework. Guoming Chen and Yuting Liao contributed equally to this work and they are co–first authors.