Multiple target tracking and stationary object detection in video with Recursive-RANSAC and tracker-sensor feedback

In this paper, we explore the concept of a tracker-sensor feedback loop, i.e., how tracker output can inform sensor processing, and investigate whether it can improve multiple target tracking performance and enable stationary object detection in video. Our implementation of the tracker-sensor feedback loop is based on the Gaussian Mixture Models (GMM) foreground detector. We modify the standard GMM algorithm by measuring target extent directly from the foreground mask and by zeroing out the background update rate of pixels associated with valid tracks. Our tracker-sensor feedback loop is incorporated into the Recursive-RANSAC framework. Recursive-RANSAC, a novel multiple target tracker, is enhanced with the inclusion of the probabilistic data association filter and nearly constant jerk motion model. We apply our algorithm to several pseudo-aerial videos that are similar to what might be captured from a UAV platform. The multiple object tracking precision and accuracy (MOTP and MOTA) and optimal sub-pattern assignment (OSPA) metrics are used to measure tracking performance. Tracker-sensor feedback is shown to produce a significant improvement in the number of missed detections, false positives, and track label switches. In terms of stationary object detection, we demonstrate our method's ability to indefinitely detect parking cars and abandoned luggage (from the PETS 2006 dataset) despite frequent occlusions and other detection challenges.