Action Recognition in Videos Using Global Descriptors and Pre-trained Deep Learning Architecture

A motion energy image (MEI) is a spatial template that collapses regions of motion into a single image in which more moving pixels are brighter than others. The forward single-step history image (fSHI) is a spatiotemporal template that shows the presence and direction of motion. Each video can be described using these templates. Recently, the advantage of deep learning architectures for human activity recognition encourages us to explore the effectiveness of combining them with these templates. Based on this opinion, a new method is proposed in this paper. Each video is split into N groups of consecutive frames, and the MEI and fSHIs are computed for each group. Transfer learning with the fine-tuning technique has been used for classifying these templates separately. By fusing the two streams of softmax scores, the final decision is acquired. This proposed method achieves the recognition accuracies of 92.60% and 93.40% for UCF Sport and UCF-11 action datasets, respectively. Also it is compared with state-of-the-art approaches and the results show that the proposed method has the best efficiency.