Multi-View Stereo Reconstruction Based on Self-Attention and Pyramid Feature Representation

Multi-view stereo (MVS) is one of the most active research topics in the field of 3D reconstruction. MVS aims to recover a 3D representation of a scene from images from multiple viewpoints and corresponding camera parameters. This paper proposes an end-to-end multi-view stereo reconstruction network by integrating a self-attention mechanism and a pyramid feature representation into the MVS network to enhance the feature extraction capability of the existing MVS network. Adding a self-attention mechanism to the U-Net network for feature extraction, output the extracted features at three scales to construct a pyramid feature map, and then infers a high-quality depth map from coarse to fine. On the basis of the obtained depth map, a depth map fusion procedure incorporating a consistency check is used to generate the final 3D point cloud model. The experimental results demonstrate that this fusion method slightly degrades the accuracy of the reconstructed 3D model compared with current representative methods, such as MVSNet, CascadeMVSNet, and PatchmatchNet, but significantly improves the completeness of the reconstructed 3D model. The proposed method is suitable for applications with strict requirements for reconstruction completeness. The experimental results on the DTU dataset show that the reconstructed completeness is improved by 1.1% relative to the base network UCSNet, which is competitive with other deep learning-based MVS methods.