Depth ablation results.

Human Activity Recognition (HAR) plays a pivotal role in video understanding, with applications ranging from surveillance to virtual reality. Skeletal data has emerged as a robust modality for HAR, overcoming challenges such as noisy backgrounds and lighting variations. However, current Graph Convolutional Network (GCNN)–based methods for skeletal activity recognition face two key limitations: (1) they fail to capture dynamic changes in node affinities induced by movements, and (2) they overlook the interplay between spatial and temporal information critical for recognizing complex actions. To address these challenges, we propose ASTM‑Net, an Activity‑aware SpatioTemporal Multi‑branch graph convolutional network comprising two novel modules. First, the Activity‑aware Spatial Graph convolution Module (ASGM) dynamically models Activity‑Aware Adjacency Graphs (3A‑Graphs) by fusing a manually initialized physical graph, a learnable graph optimized end‑to‑end, and a dynamically inferred, activity‑related graph—thereby capturing evolving spatial affinities. Second, we introduce the Temporal Multi‑branch Graph convolution Module (TMGM), which employs parallel branches of channel‑reduction, dilated temporal convolutions with varied dilation rates, pooling, and pointwise convolutions to effectively model both fine‑grained and long‑range temporal dependencies. This multi‑branch design not only addresses diverse action speeds and durations but also maintains parameter efficiency. By integrating ASGM and TMGM, ASTM‑Net jointly captures spatial–temporal mutualities with significantly reduced computational cost. Extensive experiments on NTU‑RGB + D, NTU‑RGB + D 120, and Toyota Smarthome demonstrate ASTM‑Net’s superiority: it outperforms DualHead‑Net‑ALLs by 0.31% on NTU‑RGB + D X‑Sub and surpasses SkateFormer by 2.22% on Toyota Smarthome Cross‑Subject; it reduces parameters by 51.9% and FLOPs by 49.7% compared to MST‑GCNN‑ALLs while improving accuracy by 0.82%; and under 30% random node occlusion, it achieves 86.94% accuracy—3.49% higher than CBAM‑STGCN.人类活动识别（Human Activity Recognition, HAR）在视频理解领域发挥着核心枢纽作用，其应用场景覆盖监控、虚拟现实等多个方向。骨骼数据已成为HAR任务中一种鲁棒的模态，能够克服背景噪声、光照变化等诸多挑战。然而，当前基于图卷积网络（Graph Convolutional Network, GCNN）的骨骼活动识别方法存在两大关键局限：其一，无法捕捉运动引发的节点亲和性动态变化；其二，忽视了复杂动作识别所必需的空间与时序信息间的交互作用。 为解决上述问题，本文提出ASTM-Net：一款活动感知时空多分支图卷积网络，包含两个全新模块。其一为活动感知空间图卷积模块（Activity-aware Spatial Graph convolution Module, ASGM），该模块通过融合手动初始化的物理图、端到端优化的可学习图以及动态推理得到的活动相关图，对活动感知邻接图（Activity-Aware Adjacency Graphs, 3A-Graphs）进行动态建模，从而捕捉演化的空间亲和性。其二为时序多分支图卷积模块（Temporal Multi-branch Graph convolution Module, TMGM），该模块采用通道缩减、不同扩张率的扩张时序卷积、池化以及逐点卷积等并行分支结构，可有效建模细粒度与长距离的时序依赖关系。这种多分支设计不仅能够适配多样化的动作速度与时长，还能保持参数效率。通过整合ASGM与TMGM模块，ASTM-Net能够联合捕获时空交互关系，同时大幅降低计算开销。 在NTU-RGB+D、NTU-RGB+D 120以及丰田智能家居（Toyota Smarthome）数据集上开展的大量实验验证了ASTM-Net的优越性：在NTU-RGB+D的X-Sub划分下，ASTM-Net较DualHead-Net-ALLs提升0.31%的精度；在丰田智能家居跨受试者（Cross-Subject）设置下，其性能超越SkateFormer达2.22%；与MST-GCNN-ALLs相比，ASTM-Net的参数量减少51.9%、浮点运算量降低49.7%，同时精度提升0.82%；在30%随机节点遮挡的场景下，其精度达到86.94%，较CBAM-STGCN高出3.49%。