Abandoned Luggage Detection: An Ontology-based approach for unusual activity recognition

Analysis of video data generated by surveillance systems requires an efficient way to represent, store, and retrieve for performing reasoning to identify unusual events. The recognition of unusual events is often difficult with existing machine-learning/Deep Learning approaches as they suffer due to lack of training examples. Abandoned luggage identification is one of the critical problem which poses security threat in public places. It may occur in several forms with various scenarios. However training for each possible case is extremely challenging due to limited amount of training examples. In this work, an ontology-based reasoning and analysis for identifying the complex event of left luggage in public places. A novel ontology is presented that represents the public place surveillance video data to represent various scenarios. Moreover, a reasoning is performed using Semantic Web Rule language (SWRL) for inferring relations. The proposed ontology-based approach extracts and represents salient information present in video data as a knowledge graph. The unusual events (Abandoned Luggage) is identified form the knowledge using SPARQL queries. Furthermore, the SPARQL queries can also be formulated to retrieve salient information and for question answering. The proposed framework is validated by identifying the complex events left luggage in PETS 2006, PETS 2007, AVSS 2007 and ABODA Dataset.