Evidential grid mapping, from asynchronous LIDAR scans and RGB images, for autonomous driving

We propose an evidential fusion algorithm between LIDAR scans and RGB images. LIDAR points are classified as either belonging to the ground, or not, and RGB images are processed by a state-of-the-art convolutional neural network to obtain semantic labels. The results are fused into an evidential grid to assess the drivability of an area met by an autonomous vehicle, while accounting for incoherences over time and between sensors. The dynamic behaviour of potentially moving objects can be estimated from the high-level semantic labels. LIDAR scans and images are not assumed to be acquired at the same time, making the proposed grid mapping algorithm asynchronous. This approach is justified by the need for coping with, at the same time, sensor uncertainties, incoherences of results over time and between sensors, and the need for handling sensor failure. In classical LIDAR/camera fusion, in which LIDAR scans and images are considered to be acquired at the same time (synchronously), the failure of a single sensor leads to the failure of the whole fusion algorithm. On the contrary, the proposed asynchronous approach can be used to fuse contradictory information over time, while allowing the vehicle to operate even in the event of the failure of a single sensor. Experiments on a challenging use case highlight the interest of the method.

I. Introduction

Long range 3D perception, and semantic understanding of the environment, are key challenges for mobile robots and autonomous vehicles. Although it has been proposed to fully address, at the same time, both of these issues via a multitask convolutional neural network [1], such approaches are not sufficient on their own. Convolutional neural networks seem sensible to minimal distortions within an image, as shown in [2], which justifies to only use them when paired with highly reliable sensors, such as LIDAR scanners [3]. A fusion framework generating 2D evidential grids from LIDAR scans, and segmentation results from a state-of-the-art convolutional neural network, is thus proposed in this paper. Evidential theory was adopted since this formalism can efficiently cope with incoherences between multiple pieces of information, while being able to model unobserved areas, and to manage sensor uncertainties [4]. Furthermore, 2D evidential grids are pertinent inputs for path planning systems, as in [5]. As explained in Fig. 1, Dempster's conjunctive rule is used to fuse, over time, evidential Cartesian grids from LIDAR scans, and image segmentation results projected on the ground plane. As this system presents a modular design, its components that process raw data, such as the neural network, can easily be replaced. One of the specificities of the proposed approach is its asynchronous behaviour: the LIDAR scanner and RGB camera are not assumed to acquire data at the same time. Instead, each sensor operates freely: each sensor input is processed individually, and fused with the grid representing the global state, or EgoGrid. The resulting system is flexible enough to withstand sensor failure. In the end, the main contributions of this paper are: •

A novel Cartesian grid mapping framework for LIDAR sensors;

•

A novel evidential, asynchronous, fusion scheme for semantic segmentation results generated by a convolutional neural network, and LIDAR scans;

Fig. 1:

Asynchronous fusion based algorithm. Images and LIDAR scans are individually processed once acquired. An evidential grid is then built from the sensor input, and fused with the current global grid, or egogrid, based on the current vehicle pose. Thus, no LIDAR/camera synchronization is needed.