Temporal-Spatial Deformable Pose Network For Skeleton-Based Gesture Recognition

Gesture recognition is a challenging research topic, and also has a wide range of potential applications in our daily life. With the development of hardware and advanced algorithms, we can easily extract skeleton data from video sequences and apply them for the recognition task. In this paper, we propose a novel temporal-spatial deformable pose network to leverage space and time information together. Our proposed network can automatically locate the most correlated joints across multiple frames and extract features accordingly. Additionally, we introduce a parallel multi-scale convolutional layer with different dilation rates, which can capture multi-term temporal information efficiently. We have conducted experiments on MSRC-12, ChaLearn 2013, and ChaLearn 2016 datasets and our proposed method outperforms state-of-the-art methods. Moreover, Additional experiments showed that our proposed module is more robust to handle noise data and dynamic gestures with various temporal scales.