Unaligned Hyperspectral Image Fusion via Registration and Interpolation Modeling

In satellite remote sensing, the hyperspectral sensor acquires high-spectral-resolution and low-spatial-resolution hyperspectral images (HSIs). Conversely, the multispectral sensor acquires low-spectral-resolution and high-spatial-resolution multispectral images (MSIs). Thus, HSI and MSI fusion is required to promote both spatial and spectral resolutions. Currently, most algorithms are based on the assumption that the HSI and MSI are perfectly aligned. However, this is hardly achievable in real scenarios when the two sensors acquire images from different viewpoints. In this article, we propose a fusion algorithm that consists of two stages, i.e., image registration and image fusion. For image registration, we introduce the normalized edge difference (NED) for image similarity measure considering the different resolutions of the original images. For image fusion, we incorporate the interpolation process in the spatial degradation model to compensate for the interpolation error. Experimental results show that our algorithm performs better than the state of the arts for unaligned image fusion.

I. Introduction

Hyperspectral imaging can record rich spectral information and has been applied in many fields, such as material classification [1], face recognition [2], and geological exploration [3], [4]. However, due to the limitation of physical devices, images captured by hyperspectral sensors are usually of low spatial resolution. Conversely, multispectral sensors can capture high spatial resolution but low-spectral-resolution images. Hence, many image fusion algorithms [5]–[7] have been introduced to improve both the spatial and spectral resolutions. The aim of this work is to reconstruct a high-resolution hyperspectral image (HR-HSI) by fusing the low-resolution HSI (LR-HSI) and high-resolution multispectral image (HR-MSI).