Hyperspectral Image Reconstruction From RGB Input Through Highlighting Intrinsic Properties

Dozens of spectral bands of hyperspectral images (HSIs) have been successfully reconstructed from only three color band images using deep neural networks according to their powerful nonlinear mapping capability. However, the existing deep-learning-based approaches tend to directly reconstruct HSIs from RGB inputs without emphasizing the discriminative intrinsic properties of different materials, resulting in certain distortion in reconstructed spectra. In this article, an intrinsic image decomposition (IID)-based spectral super-resolution (SSR) framework is proposed to reconstruct spectra of pixels from their reflectance feature and shading feature separately, by which the intrinsic properties can be emphasized during spectral reconstruction. Specifically, a dual hierarchical regression network (DHRNet) is designed for the proposed IID-based SSR task, in which a shading feature extraction module (SFEM) based on dense structure and a reflectance feature extraction module (RFEM) with attention mechanism are first, respectively, designed to reconstruct spectral information from reflectance feature and shading feature, and a feature enhancement module (FEM) is consequently devised to further improve the coarse combined estimation. Ultimately, a novel hybrid loss combining smooth $\boldsymbol {l}_{1}$ loss, spectral angel mapper (SAM), and gradient prior is also presented to restrain the spectral distortion while enhancing the sharpness of the reconstructed HSI. Experimental results over three datasets demonstrate the superiority of our proposed framework.

I. Introduction

Hyperspectral remote sensing (RS) images [1] are characterized by high spectral resolution, which can reach the nanometer level, providing more abundant and accurate spectral information than the conventional images. Therefore, hyperspectral RS has been widely used in various fields, including agriculture [2], [3], [4], medical imaging [5], national security [6], [7], atmospheric environment [8], and even interstellar exploration. However, these advantages of hyperspectral RS come at a great cost. On the one hand, compared with RGB images, hyperspectral image (HSI) includes two spatial dimensions and one spectral dimension, which leads to a longer exposure time. On the other hand, due to the technical and economical limitations of electro-optical imaging sensors, most spectral imaging equipment has difficulty in obtaining images with high spatial and spectral resolutions simultaneously. The fact that the spatial resolution of HSI is usually much lower than that of its RGB counterpart seriously affects the development of hyperspectral imaging applications in various fields.