Segmentation is Tracking: Spatial-Temporal Map Vehicle Trajectory Reconstruction and Validation

This paper proposed the “Segmentation is traCKing” (SiCK) solution that extract vehicle movements with semantic segmentation for high-resolution trajectory reconstruction and validation. The dynamic mode decomposition (DMD) method extracts vehicle strands by decomposing the spatial-temporal map (STMap) into the sparse foreground and low-rank background. The Res-UNet+ neural networks was designed by adapting two prevalent deep learning architectures, which significantly improved the performance of the STMap-based vehicle detection and tracking. The DMD model provides many interesting insights for understanding the evolution of underlying spatial-temporal structures preserved by the STMap. The model outputs were compared with the previous image processing model and baseline semantic segmentation neural networks. After thorough evaluations, this solution is accurate and robust against many challenging factors. Finally, this work addresses data quality issues by correcting erroneous trajectories obtained from computer vision tools. Extracting high-fidelity vehicle trajectories for transportation scientific research is a systematic process. This new framework can significantly improve the accuracy and reliability of video-based trajectory data acquisition.

I. Introduction

Video sensors have been widely used for collecting vehicle trajectory data to support academic research, traffic operations, management, and design. One of the most influential video-based trajectory datasets is the Next Generation SIMulation (NGSIM) trajectory dataset [1], which has significantly boosted traffic flow and modeling research by revealing microscopic traffic characteristics. As highlighted by a survey paper [2], although video-based trajectory datasets have greatly improved the simulations of driver behavior models and their calibration/training, there is still a substantial demand for high-quality, high-resolution trajectory data. Collecting useful trajectory data from traffic cameras with satisfactory accuracy is challenging. The traditional trajectory extraction approach, which contains multi-stage procedures, is error-prone. Previous practice for data quality control is very inefficient, involving manual processes to modify, add or delete tracked objects by hand. The signal-processing-based noise removal methods are unsatisfactory because they can only detect some incongruity points that deviate from the average driver’s behavior. However, they cannot address the root cause of detection and tracking failures. The scanline method efficiently validates extracted vehicle trajectories by showing vehicle movements on the static Spatial-temporal Map (STMap), allowing directly identifying the error for each vehicle.