Depth Map Super-Resolution Fusing Color Information

Depth estimation is an important task in many different applications such as autonomous driving and augmented reality. Obtaining accurate high resolution depth maps at high frame rates is often difficult and devices capable of doing so could be not suitable to all applications given their high price. In this work we propose two different deep learning based architectures for depth map upsampling, respectively with
$4\mathrm{x}$
and
$8\mathrm{x}$
scaling factors, with the intent of cutting costs on acquisition devices. The proposed methods also exploit as an input a high resolution color image that could be acquired simultaneously on the same scene. The proposed methods were trained and tested on two different dataset: a collection of real depth data obtained using a Kinect and the other one featuring synthetic data. Qualitative and quantitative results highlight the impact of considering high resolution color images and the ability of the proposed method to use this information in order to upsample low resolution depth maps.