SlowFast Adaptive Graph Convolutional Network with Multi-Modal Feature Aggregation for Skeleton-Based Action Recognition: Case Study of Human Ladder Climbing

In skeleton-based action recognition, the frequency of spatio-temporal changes of different joints is largely influenced by the type of action. This research presents a SlowFast Adaptive Graph Convolutional Network(SF-AGCN), which involves Slow-stream and Fast-stream networks to deal with skeleton features of different spatio-temporal changes. The Slow-stream network has low frame rate video input and high channel dimension GCNs, while the Fast-stream network has high frame rate video input and low channel dimension GCNs. On the other hand, SF-AGCN uses the spatio-temporal parallel branches and the semantic connection strength matrix to enhance the correlation between skeleton information and actions. Compared to other skeleton-based methods, SF-AGCN maintains high accuracy which is verified on both NTU-RGBD and Northwestern-UCLA datasets. It is worth noting that this model is more effective in predicting actions that occur instantaneously. In the case of human ladder climbing, SF-AGCN achieves state-of-the-art action recognition performance.