Coding Convolutional Neural Networks as Spectral Transmittance for Intelligent Hyperspectral Remote Sensing in a Snapshot

The principle and procedure of coding a convolutional neural network (CNN) in terms of the spectral transmittance of a programmable optical filter are proposed and discussed. They exhibit an intrinsic link between the CNNs and the optical filters, which leads to a methodology by which optical imaging through such spectral transmittance can be seen as equivalent to the results of hyperspectral data numerically postprocessed by the CNN. In such a manner, hyperspectral data acquisition and CNN postprocessing can be implemented simultaneously by the physical process of optical imaging in a snapshot; thus, more intelligent, informative, and real-time optical detection and sensing in the remote sensing applications can be achieved.

I. Introduction

A programmable optical filter is a kind of optical element used to multiplex the photons of different wavelengths onto a photoelectronic detector by modulating a spatial light modulator [1] or a digital micromirror device (DMD) [2], which is capable of switching its transmittance spectrum arbitrarily and rapidly. Its flexible transmittance directly enables hardware-based spectral data postprocessing by coding the spectral data postprocessing algorithms in terms of spectral transmittance, in which optical measurements with such a specific coding transmittance can be equivalent to the results of the hyperspectral data after numerical postprocessing. Compared with the conventional scanning-based hyperspectral remote sensing techniques [3], such a method only needs to collect a single image with specific coding transmittance, and the hyperspectral data acquisition and postprocessing can be implemented simultaneously by the physical process of optical imaging in a snapshot. In such a manner, the data acquisition and transmission speed can be significantly improved by at least hundreds of times compared with the scanning-based hyperspectral remote sensing techniques, and it is totally free of postprocessing. Thus, the inherent weaknesses of the hyperspectral remote sensing techniques (e.g., slow data acquisition [4], [5], mass data transmission and storage [6], and computationally expensive postprocessing [7]) can be overcome while retaining the most critical spectral information of interest for optical detection and sensing.