GabriellaV2: Towards better generalization in surveillance videos for Action Detection

Activity detection has wide-reaching applications in video surveillance, sports, and behavior analysis. The existing literature in activity detection has mainly focused on benchmarks like AVA, AVA-Kinetics, UCF101-24, and JHMDB-21. However, these datasets fail to address all issues of real-world surveillance camera videos like untrimmed nature, tiny actor bounding boxes, multi-label nature of the actions, etc. In this work, we propose a real-time, online, action detection system which can generalize robustly on any unknown facility surveillance videos. Our real-time system mainly consists of tracklet generation, tracklet activity classification, and prediction refinement using the proposed post-processing algorithm. We tackle the challenging nature of action classification problem in various aspects like handling the class-imbalance training using PLM method and learning multi-label action correlations using LSEP loss. In order to improve the computational efficiency of the system, we utilize knowledge distillation. Our approach gets state-of-the-art performance on ActEV-SDL UF-full dataset and second place in TRECVID 2021 ActEV challenge. Project Webpage: www.crcv.ucf.edu/research/projects/gabriellav2/

1. Introduction

The problem of video understanding has wide-reaching applications like action recognition [1–4], action detection [5–9], temporal action localization [10, 11], and video synthesis [12, 13].