PairingNet: A Multi-Frame Based Vehicle Trajectory Prediction Deep Learning Network

The traffic infrastructure of a city requires evaluation and improvement through a large amount of data analysis. The construction and laborious work of traditional methods make computer vision flourish in traffic analysis. Among different computer vision technologies for intelligent transportation system, one of the most important algorithms is multiple object tracking (MOT). At present, MOT used in traffic analysis has several shortcomings, such as lack of the output of vehicle speed and movement direction, and the consideration of single factor in trajectory tracking. These limitations have affected the results of traffic analysis. This research proposes an end-to-end deep learning network, called PairingNet. In addition to retaining the function and accuracy of the original detection network, PairingNet integrates the calculation of vehicle trajectory into the network through the feature fusion of consecutive images, which is introduced to predict the movement direction and speed of the vehicle. These additional features can be used to better track the trajectories of vehicles. In addition, a pipeline is designed to reduce the loading latency incurred by using the consecutive frames as the input for PairingNet. The experiment results indicate that the vehicle identification of PairingNet reaches 96% accuracy, surpassing the original YOLOv3 as the backbone structure, and reaches a near 100% accuracy rate in the predicted vehicle position. Moreover, with the pipeline process, the inference speed of PairingNet is very close to the original YOLOv3. In MOT results, the MOTA of PairingNet also has a high performance of 91%.