Human Action Recognition using ConvLSTM with GAN and transfer learning

Human Action Recognition (HAR)is a challenging time series classification problem that has received significant attention from computer vision researchers. In this paper, different techniques used for human activities are investigated, and a human action recognition approach using a Convolutional Neural Network (CNN) with Long Short Term Memory (LSTM) and generative adversarial network is proposed. The proposed research evaluates the performance of cross-entropy and adversarial loss function for HAR analysis. Two different datasets UFC101 and the classic KTH dataset, are used for experimental purposes. The UFC101 dataset contains 13k videos in which 101 human actions are included i.e., playing instrument, makeup, etc. In contrast, KTH dataset contains 600 videos containing six human activities, including walking, running, jogging, hand clapping and hand waving, performed by 25 different persons. Also, demonstrates the process of HAR by mixing both datasets and evaluate the performance. The GAN enhances the model robustness by applying adversarial training which fully discovers the underlying connections in both intra-view and cross-view aspects.