Future Frame Prediction Using Convolutional VRNN for Anomaly Detection

Anomaly detection in videos aims at reporting anything that does not conform the normal behaviour or distribution. However, due to the sparsity of abnormal video clips in real life, collecting annotated data for supervised learning is exceptionally cumbersome. Inspired by the practicability of generative models for semi-supervised learning, we propose a novel sequential generative model based on variational autoencoder (VAE) for future frame prediction with convolutional LSTM (ConvLSTM). To the best of our knowledge, this is the first work that considers temporal information in future frame prediction based anomaly detection framework from the model perspective. Our experiments demonstrate that our approach is superior to the state-of-the-art methods on three benchmark datasets.

1. Introduction

Anomaly detection is an essential problem in video surveillance. Due to the massive amount of available video data from surveillance cameras, it is time-consuming and inefficient to have human observers watching surveillance videos and report any anomalies. Ideally, we want an automatic system that can report abnormal events. Anomaly detection is challenging since the definition of “anomaly” is broad and ambiguous - anything that deviates expected behaviours can be considered as “anomaly”. It is infeasible to collect labeled training data that cover all possible anomalies. As a result, recent work in anomaly detection has focused on unsupervised approaches that do not require human labels.