Sensored Semantic Annotation for Traffic Control Based on Knowledge Inference in Video

Images and videos in multimedia data are typical representation methods that include various types of information, such as color, shape, texture, pattern, and other characteristics. Besides, in video data, information such as object movement is included. Objects may move with time, and spatial features can change, which is incorporated in spatio-temporal relations. Many research studies have been carried out over time on information recognition by computers using low-level data in this connection. There is a semantic gap between low-level and high-level information in vocabulary representing human thinking. A substantial amount of research has been conducted on reducing the semantic gap, and it is focused on representation methods of logic. The goal of this study is to understand object movement and define spatio-temporal relations through mapping between vocabulary and the object movements. Ontology mapping is a method used to bridge the gap between low-level and high-level information. In this case, the spatio-temporal relation consists of temporal relations obedient to the passage of time, directional relations obedient to changes in object movement direction, changes in object size relations, topological relations obedient to changes in object movement position, and velocity relations using concept relations between topology models. In this paper, an ontology is used to define the inference rules using the proposed spatio-temporal relations and the use of Markov Logic Networks (MLNs) for probabilistic reasoning. Finally, the performed experiment and evaluation prove the verification recognition and understanding of object movements based on video data. This paper can be extended to retrieval and comparison between object movements, automatic annotation, and video summarization. The contributions of this paper include definition of the spatio-temporal relations of a region-based object, recognition of the semantic movements of moving objects, designing and constructing a spatio-temporal ontology, and Understanding the semantic movement of moving objects.