Interaction Recognition Using Depth Information Based on 3D CNNs

The depth image sequence has better illumination insensitivity and strong texture discrimination than the traditional RGB image sequence. Although many traditional action recognition methods have achieved good performance, these approaches don't have explicit model variations and interdependencies in time, which depend on well-designed feature. The convolutional neural networks (CNNs) are a typical model of deep learning, which can achieve superior performance on action recognition without relying on handcrafted features. In this paper, we study convolutional neural networks of deep learning-based action recognition using depth image sequences. Firstly, we develop a simple 3D Convolutional Neural Networks that learn directly spatiotemporal features from raw depth image sequences. With the help of parallel computing hardware and method based on Bayesian hyper-parameter optimization, experimental results show that the designed 3D CNNs architecture is effective on simple human action recognition even complex interactive recognition and yields competitive classification performances. Secondly, we analysis the impact of different temporal extents on the accuracy of action recognition, and demonstrate that increasing the temporal extents will improve the accuracy of action recognition. Finally, in order to verify the generalization ability of the training model, we conduct a study of transfer learning by transferring the learned features to other human action datasets, experimental results show that the designed 3D CNNs architecture generalizes well and yield considerable performance improvement.