Accurate and Efficient Depth Completion Based on Attention

Depth completion is an essential task for applications such as simultaneous localization and mapping (SLAM), autonomous driving, and 3D reconstruction. It aims at restoring pixel-wise depth from sparse depth (usually captured by LiDAR or RGB-D cameras). This process can be further improved by leveraging RGB information. With multi-modality data fusion, the model is allowed to learn complementary features, thus achieving robust and accurate depth estimation. In recent years, intelligent robots and autonomous vehicles have shown great advances, but it is still challenging for them to perceive environment depth in real-time with limited hardware resources. Moreover, most existing depth completion methods choose to trade accuracy at the cost of high computational complexity, which inhibits their application in real-time robotic tasks.In this paper, we propose a deep learning-based method with attention mechanism to estimate accurate depth maps from sparse data efficiently. Specifically, we present a convolution neural network consisting of two lightweight symmetric encoder-decoder branches to extract color and depth information respectively; each branch outputs a coarse depth map and a corresponding confidence map, which are then fused adaptively. Furthermore, we perform attention-based fusion between two branches to learn multi-modality features. According to the characteristics of the depth completion problem, a loss function is designed to criteria the region with significant depth variation. This design helps to reduce blurriness and distortion in object boundaries, which is a general defect in the estimated depth map. We train and evaluate our network on the KITTI dataset. Experimental results demonstrate that our method fuses depth and color information effectively, and also estimates high-resolution depth maps with clear edges at relatively low computational cost.