PNAS-MOT: Multi-Modal Object Tracking With Pareto Neural Architecture Search

Multiple object tracking is a critical task in autonomous driving. Existing works primarily focus on the heuristic design of neural networks to obtain high accuracy. As tracking accuracy improves, however, neural networks become increasingly complex, posing challenges for their practical application in real driving scenarios due to the high level of latency. In this letter, we explore the use of the neural architecture search (NAS) methods to search for efficient architectures for tracking, aiming for low real-time latency while maintaining relatively high accuracy. Another challenge for object tracking is the unreliability of a single sensor, therefore, we propose a multi-modal framework to improve the robustness. Experiments demonstrate that our algorithm can run on edge devices within lower latency constraints, thus greatly reducing the computational requirements for multi-modal object tracking while keeping lower latency.

I. Introduction

Multiple object tracking (MOT) is a fundamental task of consistently assigning a unique ID to each observed object within a video sequence, which holds significant importance across various domains, including motion planning, safe robot navigation, and autonomous driving [1]. The primary challenge inherent to MOT lies in establishing precise associations between tracklets from preceding frames and the object detections within the current frame. To tackle the complexities of multi-object tracking, two main-stream paradigms have emerged: tracking-by-detection [2], [3] and joint-tracking-and-detection [4], [5], [6]. The tracking-by-detection paradigm follows a two-stage process. Initially, a pre-trained detector is employed to procure object detections, after which a tracker undertakes the data association task, assigning a distinct ID to each detected object across successive frames. On the other hand, the joint-tracking-and-detection paradigm endeavors to achieve detection and tracking concurrently, leveraging the benefits of joint optimization strategies. In this paper, our focus lies specifically on the tracking aspect, so we adopt the tracking-by-detection approach due to its inherent efficiency and proven effectiveness in addressing the complexities of object tracking tasks.