End-to-End Optimization of LiDAR Beam Configuration for 3D Object Detection and Localization

Existing learning methods for LiDAR-based applications use 3D points scanned under a pre-determined beam configuration, e.g., the elevation angles of beams are often evenly distributed. Those fixed configurations are task-agnostic, so simply using them can lead to sub-optimal performance. In this work, we take a new route to learn to optimize the LiDAR beam configuration for a given application. Specifically, we propose a reinforcement learning-based learning-to-optimize (RL-L2O) framework to automatically optimize the beam configuration in an end-to-end manner for different LiDAR-based applications. The optimization is guided by the final performance of the target task and thus our method can be integrated easily with any LiDAR-based application as a simple drop-in module. The method is especially useful when a low-resolution (low-cost) LiDAR is needed, for instance, for system deployment at a massive scale. We use our method to search for the beam configuration of a low-resolution LiDAR for two important tasks: 3D object detection and localization. Experiments show that the proposed RL-L2O method improves the performance in both tasks significantly compared to the baseline methods. We believe that a combination of our method with the recent advances of programmable LiDARs can start a new research direction for LiDAR-based active perception. The code is publicly available at github.com/vniclas/lidar_beam_selection.

I. Introduction

Active depth measurement systems, particularly LiDARs, are key sensors in the design of current autonomous vehicles. Due to their ability to precisely capture 3D scenes, they have been applied to a variety of robotics problems, e.g., object detection [1], depth completion [2], mapping [3], and localization [4]. Since most of these tasks benefit from a higher resolution, i.e., more measurement points, there has been an incentive in the LiDAR industry to continuously increment the number of beams of mechanical rotating scanners, currently reaching up to 128 lasers, e.g., the Velodyne Alpha Prime and RoboSense RS-Ruby.