Deep Versatile Hyperspectral Reconstruction Model from A Snapshot Measurement with Arbitrary Masks

Recently, coded aperture snapshot spectral imaging (CASSI) has been actively researched to capture three-dimensional (3D) hyperspectral (HS) images for dynamic scenes, where the optical systems detect a 2D snapshot measurement while a computational algorithm performs the inverse problem for recovering the latent HS cubic data. Benefiting from the powerful modeling capability of the deep convolution neural networks (DCNN), the reconstruction performance of the HS images has been significantly improved. However, the existing deep methods usually assume a particular hardware mask to train the reconstruction models, and restrict widely applicability to the snapshots measured in different hardwares. This study exploits a novel deep versatile HS reconstruction framework for adaptively handling the snapshots with arbitrary masks. Specifically, we employ a meta-learning like training procedure using the training paired samples of different distributions to learn a highly generalized model, and further incorporate a mask structure modeling module to produce effective knowledge for modulating the spectral recovering procedure. Moreover, we configure the deep reconstruction model with the spectral transformer for modeling the long-dependence in spectral domain, which is especially critical for high fidelity spectral recovering. Experiments on two benchmark HS datasets have demonstrated the superiority of our framework over the state-of-the-art methods.