Contrast Saliency Information Guided Infrared and Visible Image Fusion

This study proposes an infrared and visible image fusion method based on contrast saliency information guided, termed CSFusion. On the one hand, the gradual recovery network is devised to achieve self-refining processing and cross-stage integration of encoded features. Specifically, the network establishes one-to-one feature reuse between the encoder and the decoder, which can effectively facilitating the decoder integration and reconstruction of encoded features. Moreover, the introduction of the self-refine attention module (SRAM) effectively refines the encoded information of each branch and reduces the impact of redundant information on image reconstruction. On the other hand, the salient guided module (SGM) built with the edge-preserving filter can effectively output contrast saliency maps of the source images. Its synergistic operation with the objective function drives the fused image generated by the network to exhibit a better background texture while highlighting the content information of the source images. It is worth noting that the SGM as the auxiliary network is utilized only in the training stage. Compared to state-of-the-art methods, extensive experiments of qualitative and quantitative results prove the superiority and robustness of our method. Also, the performance on the object detection task further reveals its potential on high-level vision tasks.

I. Introduction

Limited by sensor hardware equipment, the image captured by a single sensor can only describe imaging scenes from a limited aspect. Image fusion technology can fuse the image from various scene into one scene, which further overcomes the limitations of the single sensor and improve the degree of scene monitoring. Moreover, image fusion, particularly multi-modal image fusion, can integrate complementary information from multiple source images. Doing so eliminates the redundancy of various sensors and maximizes the utilization of image information. This unique feature integration ability makes it suitable for various fields, including but not limited to brain tumor segmentation [1], [2], land cover mapping [3], remote sensing image detection [4], and many more tasks beyond these. Infrared and visible image fusion (IVIF) aims to generate fused images that possess the high contrast areas of infrared images while preserving the texture details of visible images. This unique capability enables the fused image to effectively mitigate the impact of external factors, such as weather conditions and variations in brightness. All these advantages have led wide its applications in the military [5], object tracking [6], [7], and video surveillance [8].