Global Cross-Attention Network for Single-Sensor Multispectral Imaging

Multispectral filter array (MSFA) imaging with one single sensor is a fast, portable, and inexpensive means of acquiring spectral images. The most challenging task for MSFA imaging is the multispectral demosaicing with the aim of reconstructing the captured raw/mosaic image, especially for the systems with many bands which results in the higher sparseness of the raw data. In this paper, a global cross-attention network (GCN) demosaicing method is proposed to excavate latent spectral characteristics to reconstruct the multispectral image. The architecture of GCN is based on the global cross-attention module, which contains a cross-transformer layer and a local self-attention module. Specifically, a global cross-attention module is proposed to fully explore intrinsic similarity characteristics in both spatial dimension and spectral dimension, non-local spatial self-attention and global spectral self-attention are conducted with Transformer architecture. Besides, the local self-attention module is utilized to enhance the effectiveness of extraction and refinement for local spatial information. Simulation experiments show that the image quality can be improved by up to 1.78 dB and the spectral similarity is significantly improved by our proposed GCN method compared to various reconstruction methods. In addition, GCN significantly reduces false color and zipper effect artifacts. Experiments using both synthetic and real data demonstrate that the proposed GCN outperforms state-of-the-art (SOTA) methods in terms of spatial and spectral fidelity.