Two-Stream Maximal Feature Attention-Guided Contrastive-Learning GAN for Image Fusion

Off-the-shelf generative adversarial network (GAN)-based image fusion methods solely utilize the diverse modal images as reference samples, aiming to make the distribution of the generated image closely resemble that of the source images through the adversarial process. This leads to the resultant images that resemble the compromised outcomes of visible images (VIs) and infrared counterparts. This article reports a seminal study on fusing infrared and VIs, which ingeniously introduces the maximal feature constraints built upon contrastive learning into a GAN-based image fusion framework, facilitating more effective semantic knowledge extraction from both image types. The generator is devised as a lateral feature aggregation codec structure, aiming to enhance interaction among multilevel features and compensate for feature loss during transmission. Unlike methods that focused only on edge information, a novel two-stream maximal feature-guided attention network upon the multilevel exclusive features is designed to strengthen the attention toward prominent information during fusion. This allows finer texture detail restoration and maintaining highlighted targets while attenuating noise. A hybrid loss function, defined as a combination of content loss and adversarial-perceptual-contrastive constraint, is minimized to generate fusion results with source imagery statistics by employing an objective that concentrates on feature distribution rather than simply evaluating pixel intensity difference. Extensive trials demonstrate that the proposed algorithm performs better than previous top-performing methods in both visual and objective aspects on four benchmark datasets.