SfM-MVS Photogrammetry with UAS: Leveraging Image Segmentation for Efficient Mapping in Dynamic Coastal Zones

Structure from Motion (SfM) photogrammetry, in conjunction with the Multi-View Stereo (MVS) technique, collectively known as SfM-MVS, emerges as a cost-effective solution for reconstructing 3D structures in real-world environments through the utilization of overlapping images. While SfM photogrammetry finds widespread use in remote sensing applications, challenges persist regarding reconstruction quality, scene segmentation, and computational complexity and efficiency. This paper introduces a workflow wherein semantically segmented images guide the SfM-MVS processing of overlapping images for reconstruction. The proposed workflow is applied to address two challenging tasks. The first task focuses on reconstructing a narrow pier situated over dynamic open ocean waves, while the second task involves simultaneous reconstruction and scene (point cloud) segmentation. Semantic labels assigned to pixels play a crucial role in determining the inclusion or exclusion of specific pixel sets during SfM-MVS processing. This experimental study underscores the promising potential of the proposed workflow to seamlessly integrate with the conventional SfM-MVS processing workflow. The approach not only augments the reconstruction quality in challenging environments but also advances the level of automation in generating spatial products within established SfM photogrammetry software suites. These findings contribute to the ongoing discourse on improving SfM-MVS methodologies for enhanced reconstruction outcomes and increased efficiency in spatial product generation.

I. Introduction

Structure-from-motion (SfM) photogrammetry is a highly efficient alternative to traditional digital photogrammetry, used to extract the geometric structure of objects or environments with intricate detail from a sequence of overlapping images. In recent years, small uncrewed aircraft systems (UASs) equipped with digital cameras have become effective tools for capturing aerial imagery in various remote sensing (RS) applications. [1].