Robust 3D Object Tracking from Monocular Images Using Stable Parts

We present an algorithm for estimating the pose of a rigid object in real-time under challenging conditions. Our method effectively handles poorly textured objects in cluttered, changing environments, even when their appearance is corrupted by large occlusions, and it relies on grayscale images to handle metallic environments on which depth cameras would fail. As a result, our method is suitable for practical Augmented Reality applications including industrial environments. At the core of our approach is a novel representation for the 3D pose of object parts: We predict the 3D pose of each part in the form of the 2D projections of a few control points. The advantages of this representation is three-fold: We can predict the 3D pose of the object even when only one part is visible; when several parts are visible, we can easily combine them to compute a better pose of the object; the 3D pose we obtain is usually very accurate, even when only few parts are visible. We show how to use this representation in a robust 3D tracking framework. In addition to extensive comparisons with the state-of-the-art, we demonstrate our method on a practical Augmented Reality application for maintenance assistance in the ATLAS particle detector at CERN.

1 Introduction

Methods for 3D object detection and tracking have undergone impressive improvements in recent years  [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14]. However, each of the current approaches has its own weaknesses: Many of these approaches [1], [3], [9], [13] rely on a depth sensor, which would fail on metallic objects or outdoor scenes; methods based on feature points [6], [8] expect textured objects; those based on edges  [4], [7] are sensitive to cluttered background; most of these methods [2], [3], [5], [10], [11], [12], [15] are not robust to occlusion. We also want a method fast enough for interactive 3D applications.