A Knowledge Optimization-Driven Network With Normalizer-Free Group ResNet Prior for Remote Sensing Image Pan-Sharpening

Multispectral (MS) images play a crucial role in environmental monitoring or ecological analysis for their large scope, quick acquisition, and big data. With the rapid development of technology and increasing demand, very high-resolution MS images have attracted a lot of attention these days. However, due to sensor equipment and the imaging environment, the spatial resolution of MS images is always restricted. With the help of panchromatic images, pan-sharpening is a very important technique to enhance the spatial details of MS images. In this study, we proposed a knowledge optimization-driven pan-sharpening network with normalizer-free group ResNet prior, called PNXnet, which is unfolded from a physical knowledge optimization-driven variational model. We solved the memory overhead brought by the traditional ResNet relying on batch normalization. Results on four sensors show that high quantitative indexes and natural visual effects have verified the reliability of PNXnet. Focusing on the near-infrared (NIR) band where spatial details are hard to be injected, we compared the normalized difference vegetation index (NDVI) generated from the fused results, the estimated NDVI shows a high consistency to the ground truth with $R^{2}$ above 0.91. Besides, we also compared the model generation. Furthermore, low model complexity and quicker computational speed make the daily application of PNXnet possible.

I. Introduction

Remote sensing technique, with the merit of huge observation scope, abundant information, and fixed revisit period, has been of extraordinary significance to many fields, including hydrometeorology [1], agriculture [2]–[4], and environmental monitoring [5]–[7]. Meanwhile, images collected by remote sensing satellites always represent Earth’s surface from two aspects, namely spectral and spatial dimensions. Spectral information is beneficial to recognize ground objects by representing the physical property, while spatial detail makes great sense to finer applications, both of which are of great importance [8].