Reconstruction of Large-Scale Missing Data in Remote Sensing Images Using Extend-GAN

Numerous studies have been conducted on missing data recovery in remote sensing images, such as cloud removal and dead pixels restoration. Nevertheless, reconstructing continuous, extensive, and complete missing areas still poses a significant challenge. In this letter, we propose a new architecture named Extend-generative adversarial network (GAN), which leverages only a low-resolution image with relaxed requirements on spatial resolution and acquisition time as a condition to reconstruct a high-resolution image with large-scale missing areas. We equip Extend-GAN with learnable adaptive region normalization (LARN) to adjust the intensity distribution of pixels to reduce color distortion. We also introduce a new loss function into the training process of Extend-GAN, namely the structural similarity (SSIM)-based triplet loss, which helps to preserve the between missing parts and known regions. Gaofen-2 and Landsat-9 image pairs are used to validate the proposed method. Extend-GAN performs better when comprehensively evaluated on visual effect, quantitative metrics, processing speed, etc. Code is available at https://github.com/yc-cui/Extend-GAN.

I. Introduction

Missing data recovery in remote sensing images is a classical yet challenging task. Many applications, such as recovering the images of Landsat Enhanced Thematic Mapper Plus (ETM+) (scan line corrector (SLC)-off), repairing the occluded areas of clouds and shadows, or filling the region in mosaic of large-scale images, etc., are often regarded as missing data recovery problems. The nature of these problems is to estimate the missing areas and fill the vacancies with predicted pixels so that the remedied image looks visually and semantically correct and the data usability is also improved.