Crime Detection from Pre-Crime Video Analysis with Augmented Activity Information

This study focuses on the detection of pre-crime events in videos, specifically shoplifting. In video understanding, visual features, pose, and emotion information are inherently frame-centric, providing insights in understanding within isolated frames. Our approach introduces a novel method by extracting activity information between frames and incorporating visual features augmented with human pose and emotion information to enhance the understanding of pre-shoplifting behaviors. We used a set of CCTV videos in which at the end of some clips the customers shoplift and in others they do not. We used these videos as training data with a transformer machine learning architecture. We augmented low level video analysis with pose, and emotion information on frame level and extracted activity information between frames. We conducted a systematic series of experiments to assess the performance of our model under two distinct configurations. The first configuration involved frame-centric visual features augmented with pose and emotion information, while the second configuration focused on activity information between frames. The results demonstrate improvements, underscoring the pivotal role of extracting temporal dynamics between frames in capturing nuanced activities leading up to the crime. The comprehension and capture of these activity patterns between frames proved to be crucial for a more thorough examination of pre-shoplifting events. The essence of our study's contribution is hinged upon the utilization of extracting temporal dynamics between frames, enabling the capture of crucial activities of individuals prior to engaging in shoplifting.