Detail-Optimized Super-Resolution Reconstruction-Based Multistage Training Strategy for Remote Sensing Semantic Segmentation

Low resolution is a major factor that negatively impacts the accuracy of remote sensing (RS) interpretation. High-quality super-resolution reconstruction (SRR) can help alleviate this problem. In this study, we present a multistage semantic segmentation training strategy called SRSSTS, which is based on detail-optimized SRR. SRSSTS addresses the challenges of poor reconstruction quality and low semantic segmentation (SS) accuracy of RS images. Our approach involves constructing a generative adversarial network (GAN)-based perceptual-loss-dominated SRR network, which generates texture-realistic, high-quality high-resolution images from low-resolution RS images. We then input the generated images into an SS network in steps and propose a three-stage joint loss to generate segmentation results at different resolutions. SRSSTS is straightforward and efficient, and it can be applied to any SS backbone networks. Additionally, this strategy does not increase computational resources compared to other SR-based segmentation methods, since the SRR network is trained separately. We evaluated the effectiveness of our designed SRR network and SRSSTS on five RS datasets (ISPRS Potsdam and Vaihingen, WHDLD, LoveDA Rural and Urban) with different resolutions, achieving excellent performance compared to an SS model for a single task and other SR-based segmentation methods. Moreover, we observed that the learned perceptual image patch similarity (LPIPS) of the super-resolution (SR) reconstructed images, i.e., the stronger the perception ability, the more helpful it is for the SS task.