Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization

Fusing a low spatial resolution hyperspectral image (LR-HSI) with a high spatial resolution multispectral image (HR-MSI) to obtain a high spatial resolution hyperspectral image (HR-HSI) has attracted increasing interest in recent years. In this paper, we propose a coupled sparse tensor factorization (CSTF)-based approach for fusing such images. In the proposed CSTF method, we consider an HR-HSI as a 3D tensor and redefine the fusion problem as the estimation of a core tensor and dictionaries of the three modes. The high spatial-spectral correlations in the HR-HSI are modeled by incorporating a regularizer, which promotes sparse core tensors. The estimation of the dictionaries and the core tensor are formulated as a coupled tensor factorization of the LR-HSI and of the HR-MSI. Experiments on two remotely sensed HSIs demonstrate the superiority of the proposed CSTF algorithm over the current state-of-the-art HSI-MSI fusion approaches.

I. Introduction

Hyperspectral imaging is an emerging modality where a camera acquires images from a scene across a number of different wavelengths. The very high spectral resolution and coverage of hyperspectral images (HSIs) enable a precise identification of the materials present in the scene, which underlies a large number of remote sensing [1]–[4] and computer vision applications [5], [6]. However, since the photons emitted by the sun are spread over many spectral bands, the spatial resolution has to be decreased in order to maintain the number of photons in each band above a minimum value. In this way, the signal-to-noise-ratio (SNR) due to the Poisson noise is kept above a minimum acceptable value [7]. As a result, the spatial resolution of HSIs is often poor. Compared with hyperspectral imaging sensors, the existing multispectral imaging sensors capture MSIs with much higher spatial resolution and SNR [8]. Therefore, the low spatial resolution hyperspectral images (LR-HSIs) are often fused with high spatial resolution multispectral images (HR-MSIs) to reconstruct hyperspectral images with high spatial resolution (HR-HSIs). This procedure is referred to as HSI super-resolution or HSI-MSI fusion.