VizOPS: A data-driven ontology to represent public place surveillance data

Large-scale knowledge graphs such as Wikidata and DBpedia have become a powerful asset for semantic search, question answering and extraction of semantic information. However, most of the data in such knowledge-base(s) is based on the textual data with very emphasis on the visual data. Moreover, due to rapid deployment of surveillance systems in public places, massive amount of video data is being produced. Storage and analysis of this data in real time is a substantial challenge. Therefore, there is massive potential in representing the visual data in knowledge graph due to presence of hidden semantics present in images and videos. However, constructing knowledge graph requires ontology, which needs expertise and domain knowledge. In this paper, we present a novel data-driven ontology (“VizOPS”) to represent the visual information of public-place surveillance data to automate ontology engineering process. The ontology is constructed by using the existing public domain benchmark datasets and its metadata. The Class for the ontology is obtained by the existing class from the various public place surveillance data. The relation between two classes is extracted using NLP-based algorithm image captioning datasets. This way the ontology is able to capture the most of the salient surveillance information of real-world public places. The proposed ontology is evaluated by representing the concepts of public places using Visual Genome dataset.