A Coarse-to-Fine Deep Generative Model With Spatial Semantic Attention for High-Resolution Remote Sensing Image Inpainting

Large area array CCD splicing technology and hardware damage lead to the lack of information in high-resolution remote sensing images (HRSIs), which seriously restricts the application of remote sensing images. However, the complex spatial geographical relationship and the serious spatial heterogeneity will cause texture blur and incomplete structure, especially in the boundary and highly textured regions of HRSI. To deal with the challenge, this article proposes a coarse-to-fine deep generative model with a novel spatial semantic attention (SSA) mechanism. SSA is deployed in the encoder and decoder of the network to ensure the continuity of local features and the relevance of global semantic information and embedded in fine network. In the coarse network, a multilevel loss is proposed to provide more accurate information. Fine network performs more detailed image inpainting operations based on coarse network. Extensive experiments on three datasets, including GF2 satellite images, aerial images, and natural ones, demonstrate that our proposed model has better inpainting effect than the state-of-the-art algorithms.

I. Introduction

High-resolution remote sensing images (HRSIs) contain rich details of ground features and are important carriers for the effective expression of geographic information. In recent years, with the development of optical imaging technology, optical system design continues to toward high resolution and a large field of view in order to obtain more details and a wide spatial range of target image information [1]–[3]. Limited by the CCD manufacturing process and the difficulty of splicing, when splicing large area array focal planes, mechanical direct splicing is generally used [4]–[11].