Activity Recognition Based on FR-CNN and Attention-Based LSTM Network

A human activity recognition (HAR) based on the Faster Region-based Convolutional Neural Network (FR-CNN) and attention-based LSTM networks is proposed in this paper. A new structure of posture vector is generated by extracting skeleton joints of human movement using the pre-trained FR-CNN model. The Cornell Activity Dataset (CAD-60) is employed in the training and test phases. An attention-based bidirectional LSTM (Bi-LSTM) network is presented for activity classification. Experimental result shows that the attention-based Bi-LSTM network achieves the precision and recall rate of 97.02% and 96.83%, respectively, in recognizing twelve activities. The result is superior to the other existing systems, indicating the application potential of our work.