Learning Scribbles for Dense Depth: Weakly Supervised Single Underwater Image Depth Estimation Boosted by Multitask Learning

Estimating depth from a single underwater image is one of the main tasks of underwater visual perception. However, data-driven underwater depth estimation methods have long been challenging to make breakthroughs due to the difficulty in obtaining a large number of true-value references. This is partly due to the high cost of acquisition equipment, which is difficult to be applied to diverse ocean scenes by a wide range of users, and therefore, sample diversity is difficult to guarantee; on the other hand, manual annotation of dense depth relationships is almost impossible to achieve. In this article, we establish a new underwater relative depth estimation benchmark, namely, segmentation of underwater imagery (SUIM)-sparse depth annotation (SDA), by extending the SUIM dataset with more than 6000 manually annotated depth trendlines, 25 million pixels with paired depth-ranking labels, and 14 million depth-ranked pixel pairs. Using the sparse depth relationship annotation provided by SUIM-SDA and the semantic information provided by SUIM, we design a new multistage multitask learning framework to predict a dense relative depth map for a single underwater image. Comprehensive comparison and ablation study on the publicly available dataset and our new benchmark demonstrate the effectiveness of the proposed weakly-supervised strategy for dense relative depth estimation. The new benchmark, source code, and trained models are available on the project home page: https://wangxy97.github.io/WsUIDNet.

I. Introduction

In recent years, more and more underwater mobile observation devices, such as remote operated vehicles (ROVs) and autonomous underwater vehicles (AUVs), are equipped with high-performance vision sensors, and vision-based underwater remote sensing technology is receiving more and more attention from [1], [2], [3], [4], and [5]. Depth estimation from a single underwater image is one of the fundamental tasks of underwater visual perception. It is important for the analysis, understanding, and even reconstruction of underwater scenarios. Thanks to the rapid development of deep learning techniques, the problem of monocular depth estimation on land has been extensively studied and has made great progress [6], [7], [8], [9], [10]. While, due to the specificity of the underwater environment, it is difficult and costly to use time-of-flight (ToF) [11] strategies and structured light sensors [12] to collect real-world dense depth information and thus build large general underwater depth datasets. Therefore, supervised learning, such as deep regression with convolutional neural networks (CNNs), cannot be performed directly for underwater scenarios such as the land-based depth estimation task.