Repair of Holes Appearing in the Stereo Reconstruction Based on 3D Reprojection and DCGAN

With the development of digital 3D city construction, the use of multi-visual stereo method for 3D reconstruction of buildings has received a lot of attention and research. OpenMVG and OpenMVS are a common set of tools among them, due to their open-source code and high integrity. These tools complete the 3D reconstruction by recovering the structure from motion (SFM) of the target and then performing mesh reconstruction, texture mapping and other processes. However, due to incomplete shooting angles and occlusion problems, some holes exist on the surface of the reconstructed object, which affects the aesthetics and integrity of the result. To address this problem, this paper proposes a hole repair method based on 3D reprojection and DCGAN, to optimize the reconstruction results by integrating 3D structure repair and 2D texture repair. The holes contained in the reconstructed 3D target are filled by interpolation. For the obtained 2D imaging map containing holes, the texture values of the holes are recovered by introducing DCGAN generating adversarial networks, that are finally mapped to the 3D structure. The experimental test results indicate that the method can effectively restore the holes’ 3D information and texture speculation, which results in a more complete 3D reconstruction model of the building.