LIV-DeepSORT: Optimized DeepSORT for Multiple Object Tracking in Autonomous Vehicles Using Camera and LiDAR Data Fusion

Object detection and tracking play a crucial role in the perception systems of autonomous vehicles. Simple Online Real-Time (SORT) techniques, such as DeepSORT, have proven to be among the most effective methods for multiple object tracking (MOT) in computer vision due to their ability to balance high performance with robustness in challenging scenarios. This article presents a method for adapting and optimizing the DeepSORT tracking algorithm to meet the demands of autonomous driving applications. Our approach leverages the Mask-Mean algorithm [2] to fuse data from cameras and LiDARs, as well as to detect, segment, and extract the 3D positions of objects in real-world space. In objects tracking, we take into account the ego-vehicle’s motion to estimate each object’s state, and the Unscented Kalman Filter (UKF) is utilized to handle the nonlinearity of each object’s motion state in real-world space. Our optimized version of DeepSORT, known as LIV-DeepSORT, demonstrates its ability to track multiple objects with high levels of robustness and accuracy, even in dynamic environments, making it suitable for the perception systems of autonomous vehicles.