ConvBKI: Real-Time Probabilistic Semantic Mapping Network With Quantifiable Uncertainty

In this article, we develop a modular neural network for real-time (>10 Hz) semantic mapping in uncertain environments, which explicitly updates per-voxel probabilistic distributions within a neural network layer. Our approach combines the reliability of classical probabilistic algorithms with the performance and efficiency of modern neural networks. Although robotic perception is often divided between modern differentiable methods and classical explicit methods, a union of both is necessary for real-time and trustworthy performance. We introduce a novel convolutional Bayesian kernel inference (ConvBKI) layer which incorporates semantic segmentation predictions online into a 3-D map through a depthwise convolution layer by leveraging conjugate priors. We compare ConvBKI against state-of-the-art deep learning approaches and probabilistic algorithms for mapping to evaluate reliability and performance. We also create a robot operating system package of ConvBKI and test it on real-world perceptually challenging off-road driving data.

I. Introduction

Robotic perception is at a crossroads between classical, probabilistic methods and modern, implicit deep neural networks. Classical probabilistic solutions offer reliable and trustworthy performance at the expense of longer run-times and unoptimized, hand-tuned parameters. In contrast, modern deep learning methods achieve higher performance and improved latency due to optimization in an implicit space but lose the ability to generalize to new data. However, probabilistic and deep learning methods are not intrinsically opposed and may be combined through methodical design.