DisARM: Displacement Aware Relation Module for 3D Detection

We introduce Displacement Aware Relation Module (DisARM), a novel neural network module for enhancing the performance of 3D object detection in point cloud scenes. The core idea is extracting the most principal contextual information is critical for detection while the target is incomplete or featureless. We find that relations between proposals provide a good representation to describe the context. However, adopting relations between all the object or patch proposals for detection is inefficient, and an imbalanced combination of local and global relations brings extra noise that could mislead the training. Rather than working with all relations, we find that training with relations only between the most representative ones, or an-chors, can significantly boost the detection performance. Good anchors should be semantic-aware with no ambiguity and able to describe the whole layout of a scene with no redundancy. To find the anchors, we first perform a preliminary relation anchor module with an objectness-aware sampling approach and then devise a displacement based module for weighing the relation importance for better utilization of contextual information. This lightweight relation module leads to significantly higher accuracy of object instance detection when being plugged into the state-of-the-art detectors. Evaluations on the public benchmarks of real-world scenes show that our method achieves the state-of-the-art performance on both SUN RGB-D and Scan-Net V2. The code and models are publicly available at https://github.com/YaraDuan/DisARM.

1. Introduction

Detecting objects directly from the 3D point cloud is challenging yet imperative in many computer vision tasks, such as autonomous navigation, path planning for robotics, as well as some AR applications. The goal of 3D object detection is to localize all valid shapes and recognize their semantic label simultaneously, which puts forward high requirements for understanding the whole input scene. Figure 1.

Illustration of the importance of DisARM. (b) It is easy to mistake the cabinet as a table when the point cloud is incomplete and featureless. (c) Redundant relations are usually incomplete and lose the important displacement information of the target object. (d) The network can recognize and locate the cabinet easily with the help of DisARM which provides valid surrounding environment information.