Multimodal Sensors Image Fusion for Higher Resolution Remote Sensing Pan Sharpening

Panchromatic (PAN) sensors and multispectral (MS) sensors are usually applied to remote sensing tasks. Pan sharpening is an effective image fusion method based on multimodal sensors. In pan sharpening, high-resolution (HR) PAN images from PAN sensors and corresponding low-resolution MS images from MS sensors are used to obtain the HR MS images. Normally, the ideal pan-sharpening image is a fused MS image with the same resolution as the PAN image. We hope to obtain the pan-sharpening images with higher resolution, which will not only preserve spatial structure information and spectral information from PAN images and MS images but also achieve super-resolution reconstruction. For this purpose, a novel pan-sharpening framework, called multilevel and multiscale fusion network (MLMSFN), is constructed. To our knowledge, our framework is the first work to achieve beyond the limit of existing image resolution in pan sharpening. We evaluate the effectiveness of our designed method, and the experiments testify that our method can obtain higher resolution pan-sharpening images.

I. Introduction

Remote sensing images are obtained from satellite sensors, which are applied on the ground users after image preprocessing and compression transmission. These images are used in various applications, such as agrometeorology, military reconnaissance, mining, and environmental monitoring. Considering the application characteristics of remote sensing images, the demand for multispectral (MS) remote sensing images is increasing with the development of the data era. MS images store multiple images corresponding to each band in the spectrum. The information of each band has high application value. However, with the limitation of imaging devices, the reality is that the resolution of MS images obtained from satellites cannot satisfy the application requirements. In order to solve this problem, satellites usually use multiple sensors to obtain remote sensing images from the same scene and then use the information of these different images to produce satisfactory MS images. These images usually include high resolution (HR) but only single-channel panchromatic (PAN) images and low-resolution (LR) MS images with the corresponding scenes. Pan-sharpening technology can fuse the spatial structure information of HR-PAN images with the spectral information of LR-MS images and ultimately preserve this information as much as possible to obtain HR-MS images [1], [2].