MSCDGCN: Multi-scale central difference graph convolution network for skeleton-based action recognition

Abstract

Effectively distinguishing fine-grained actions remains a critical challenge in skeleton-based action recognition. Existing Graph Convolutional Network methods often overlook directional motion cues and fail to integrate spatiotemporal features efficiently. To address these limitations, this paper proposes a novel Multi-Scale Central Difference Graph Convolutional Network (MSCDGCN) for skeleton-based action recognition. This model introduces a sparse self-attention-based central difference graph convolution that highlights key joints, enhancing local feature extraction while capturing contextual dependencies and intrinsic skeletal topology. A Spatial Temporal Joint Focus (STJF) module is designed to efficiently fuse spatial features extracted by the self-attention central difference graph convolution with temporal features obtained through Multi-Scale Separable Temporal Convolution (MSSTC). Experiments on NTU RGB + D 60 and NTU RGB + D 120 demonstrate state-of-the-art performance. Specifically, MSCDGCN achieves accuracies of 92.8% (X-Sub) and 96.8% (X-View) on NTU RGB + D 60, surpassing previous methods. On the larger NTU RGB + D 120 dataset, it attains accuracies of 89.5% (C-Sub) and 91.0% (C-Set). Furthermore, the recognition accuracy rate on the cross-dataset Kinetics is recorded at 39.8%, thereby validating its performance advantage. Ablation studies confirm the contributions of each module: SACD and STJF collectively enhance accuracy by 1.0% through directional feature learning and spatiotemporal fusion mechanisms. These results show that MSCDGCN effectively tackles fine-grained recognition and computational efficiency. This framework offers a robust solution for real-world applications necessitating precise motion differentiation, such as surveillance systems and human–computer interaction scenarios.