Learning Depth from Monocular Videos Using Direct Methods

The ability to predict depth from a single image - using recent advances in CNNs - is of increasing interest to the vision community. Unsupervised strategies to learning are particularly appealing as they can utilize much larger and varied monocular video datasets during learning without the need for ground truth depth or stereo. In previous works, separate pose and depth CNN predictors had to be determined such that their joint outputs minimized the photometric error. Inspired by recent advances in direct visual odometry (DVO), we argue that the depth CNN predictor can be learned without a pose CNN predictor. Further, we demonstrate empirically that incorporation of a differentiable implementation of DVO, along with a novel depth normalization strategy - substantially improves performance over state of the art that use monocular videos for training.

1. Introduction

Depth prediction from a single image using CNN s has had a surge of interest in recent years [7], [21], [20]. Recently, unsupervised methods that rely solely on monocular video for training (without depth or stereo groundtruth) have captured the attention of the community. Of particular note in this regard is the work of Zhou et al. [31] who proposed a strategy that learned separate pose and depth CNN predictors by minimizing the photometric consistency across monocular video datasets during training. Although achieving impressive results this strategy falls noticeably behind those that have been trained using rectified stereo image pairs [13], [19]. These rectified stereo methods have shown comparable accuracy to supervised methods [26], [7], [23] over datasets where only sparse depth annotation is available. However, the assumption of using calibrated binocular image pairs excludes itself from utilizing monocular video which is easier to obtain and richer in variability. This performance gap between stereo [13], [19] and monocular [31] learning strategies is of central focus in this paper.