HerosNet: Hyperspectral Explicable Reconstruction and Optimal Sampling Deep Network for Snapshot Compressive Imaging

Hyperspectral imaging is an essential imaging modality for a wide range of applications, especially in remote sensing, agriculture, and medicine. Inspired by existing hyperspectral cameras that are either slow, expensive, or bulky, reconstructing hyperspectral images (HSIs) from a low-budget snapshot measurement has drawn wide attention. By mapping a truncated numerical optimization algorithm into a network with a fixed number of phases, recent deep unfolding networks (DUNs) for spectral snapshot compressive sensing (SCI) have achieved remarkable success. However, DUNs are far from reaching the scope of industrial applications limited by the lack of cross-phase feature interaction and adaptive parameter adjustment. In this paper, we propose a novel Hyperspectral Explicable Reconstruction and Optimal Sampling deep Network for SCI, dubbed HerosNet, which includes several phases under the ISTA-unfolding framework. Each phase can flexibly simulate the sensing matrix and contextually adjust the step size in the gradient descent step, and hierarchically fuse and interact the hidden states of previous phases to effectively recover current HSI frames in the proximal mapping step. Simultaneously, a hardware-friendly optimal binary mask is learned end-to-end to further improve the reconstruction performance. Finally, our HerosNet is validated to outperform the state-of-the-art methods on both simulation and real datasets by large margins. The source code is available at https://github.com/jianzhangcs/HerosNet.

1. Introduction

With the development of artificial intelligence and robotics, the demand for capturing and sensing hyper- spectral images has dramatically increased in recent years [3], [6], [12], [47]. Based on the traditional compressive sensing (CS) [51], [54], spectral snapshot compressive sensing (SCI) system aims to record 3D scenes via a 2D detector. It has the advantages of low bandwidth, low cost, and high data throughput, which has played an increasingly pivotal role in a wide range of applications, such as remote sensing, object detection, super-resolution, and medical diagnosis [2], [8], [14], [18], [25], [36], [38], [55]. In this paper, we focus on a typical imaging system named coded aperture snapshot spectral imager (CASSI) [11], [21], [30], which modulates spectral frames via a coded aperture (i.e. physical mask) and shifts them across the spectral dimension via a disperser. Figure 1.

The schematic of CASSI system and some visual results of the proposed HerosNet and DGSM [13] on the real dataset. Our reconstructed HSIs have clearer edges and more detailed textures, while the results of DGSM have more noise and artifacts.