Spatiotemporal-Spectral Graph Convolutional Networks For Skeleton-Based Action Recognition

Skeleton-based action recognition has been a hot topic with the increasing development of Graph Convolutional Networks(GCNs). Previous work constructed the skeleton sequences into graphs and focused on extracting spatial-temporal information from various actions. However, they ignored the hidden information in the spectral domain of the whole graphs. In this paper, a novel graph network based on both spatio-temporal information and spectral-domain information is proposed(SS-GCN), adopting a two-stream graph topology and can be trained in an end-to-end manner. Besides, along with other GCN methods that optimize only the spatial-temporal graph, our spectral stream helps in further performance improvements. Our method(SS-GCN) is evaluated on two large skeleton-based datasets, NTU-RGBD and Kinetics-Skeleton. The experiment results demonstrate the effectiveness of SS-GCN.