Context-Based Multiscale Unified Network for Missing Data Reconstruction in Remote Sensing Images

Missing data reconstruction is a classical yet challenging problem in remote sensing images. Most current methods based on traditional convolutional neural network require supplementary data and can only handle one specific task. To address these limitations, we propose a novel generative adversarial network-based missing data reconstruction method in this letter, which is capable of various reconstruction tasks given only single source data as input. Two auxiliary patch-based discriminators are deployed to impose additional constraints on the local and global regions, respectively. In order to better fit the nature of remote sensing images, we introduce special convolutions and attention mechanism in a two-stage generator, thereby benefiting the tradeoff between accuracy and efficiency. Combining with perceptual and multiscale adversarial losses, the proposed model can produce coherent structure with better details. Qualitative and quantitative experiments demonstrate the uncompromising performance of the proposed model against multisource methods in generating visually plausible reconstruction results. Moreover, further exploration shows a promising way for the proposed model to utilize spatio-spectral-temporal information. The codes and models are available at https://github.com/Oliiveralien/Inpainting-on-RSI.

I. Introduction

Driven by the rapid development of satellite technology, remote sensing image processing has been receiving more and more attention. However, due to the satellite sensor working conditions and the atmospheric environment, remote sensing images often suffer from missing information problems, such as dead pixels, cloud or shadow removal, as shown in Fig. 1. These problems obscure land surface features, leading to adverse effects to subsequent image processing such as classification [1], detection [2], and segmentation [3]. Therefore, recovering the inadequate information of remote sensing imagery has become an urgent need. Fig. 1.

Three representative missing data problems of remote sensing data.