A Lightweight and Detector-Free 3D Single Object Tracker on Point Clouds

Recent works on 3D single object tracking treat the task as a target-specific 3D detection task, where an off-the-shelf 3D detector is commonly employed for the tracking. However, it is non-trivial to perform accurate target-specific detection since the point cloud of objects in raw LiDAR scans is usually sparse and incomplete. In this paper, we address this issue by explicitly leveraging temporal motion cues and propose DMT, a Detector-free Motion-prediction-based 3D Tracking network that completely removes the usage of complicated 3D detectors and is lighter, faster, and more accurate than previous trackers. Specifically, the motion prediction module is first introduced to estimate a potential target center of the current frame in a point-cloud-free manner. Then, an explicit voting module is proposed to directly regress the 3D box from the estimated target center. Extensive experiments on KITTI and NuScenes datasets demonstrate that our DMT can still achieve better performance ( $\sim $ 10% improvement over the NuScenes dataset) and a faster tracking speed (i.e., 72 FPS) than state-of-the-art approaches without applying any complicated 3D detectors. Our code is released at https://github.com/jimmy-dq/DMT.

I. Introduction

Single object tracking (SOT) is a key task in the field of computer vision, which has wide downstream applications in outdoor and indoor scenarios, ranging from autonomous driving [1], [2], robot vision [3], [4], [5], [6], and intelligent transportation systems [7]. For example, an autonomous pedestrian-following robot should accurately track its master for efficient crowd-following control. Another example is autonomous landing by unmanned aerial vehicles, in which the drone must track the target and know the exact distance and pose of the target in order to land safely [8]. In indoor environments, tracking methods [5], [6], [9] can provide the six-degrees-of-freedom (6DoF) pose of an object for robust robotics manipulation. Given an initial bounding box of a template object in the first frame from images or LiDAR scans, the aim of SOT is to estimate its location by identifying the trajectory across all frames. In the past decade, a variety of image-based trackers (e.g., Siamese neural networks [10]) have shown promising performance in the 2D tracking community. However, the performance of image-based methods often suffers in degraded situations, e.g., when facing drastic lighting changes [11], [12]. As a possible remedy, 3D point clouds collected from LiDAR provide detailed depth and geometric information, which is inherently invariant to lighting changes [13], making it more robust when tracking across frames taken from different illumination environments.