Semantic labeling of track events using time series segmentation and shape analysis

This paper presents a novel framework for applying semantic labels to events within a track. A track is a two-dimensional (2D) or a three-dimensional (3D) signal in time where each point of the signal is the x and y (and z) centroid spatial coordinate of an object at a specific frame of the video. The track may be generated by the movement of a vehicle, person, or object. In the 2D case, the signal is decomposed into x and y time series for use in one-dimensional time series segmentations. Then the results of the two segmentations are combined to produce a 2D signal segmentation of the track which results in unique events to be labeled. The Procrustes measure, from shape analysis, is employed along with template matching to find the most likely trajectory of each individual event. Once each event is labeled with a semantic description from the template, we enhance the label using other basic measurements based on the track. The application of our framework on 4 vehicle tracks from original videos is shown to display the efficacy of our method.

1. INTRODUCTION

Much attention has been paid to methods for identifying atomic actions of a vehicle (or other object) in a video scene, such as if a right or left turn has been made. Previous methods of event labeling have used statistical shape theory and Autoregressive and Moving Average (ARMA) for activity recognition [1]. However, little attention has been paid to trying to semantically describe a more complex vehicle track or path. Unmanned aerial vehicles (UAV), global positioning satellite (GPS) tracking devices, and other sensors have the ability to track vehicles over a long period of time recording many complex activities. Currently, human operators are responsible for describing an object's actions in a complex video track. We propose a framework for automatically segmenting an object's track into meaningful events and applying semantic labels to those events within a given track.