Self-Supervised Depth Estimation Based on the Consistency of Synthetic-real Image Prediction

Learning-based multi-view stereo aims to restore the real scene from multiple images with overlapping areas. The mainstream self-supervised MVS method trains the model based on the assumption that spatial points from different perspectives share the same color information. To further suppress the interference from specular reflection and illumination noise, this work proposes a self-supervised MVS network based on the consistency of synthetic-real image prediction. The network first applies the coarse-to-fine manner to gradually refine the depth map, and the source images are projected to the reference view to generate the synthesized reference image. Then the synthesized image with real source images are re-input into the network to form a cycled network, and the consistency constraint of the prediction results of the two periods is introduced to improve the color anti-interference of the self- supervised MVS model. The comprehensive experiments on the public dataset show that the proposed work can further improve the reconstruction performance of the benchmark model, which verifies the effectiveness of the proposed work.