RCAFusion: Cross Rubik Cube Attention Network for Multi-modal Image Fusion of Intelligent Vehicles

Multi-modal fused images can provide reliable perceptual information for intelligent vehicles in various weather and lighting conditions. However, most existing fusion algorithms neglect the information interactions among different modalities, leading to a loss of essential information in transportation systems characterized by strong information correlations. To enhance the quality of multi-modal semantic information fusion perception in intelligent vehicles, we propose the Cross Rubik Cube Attention Fusion Network (RCAFusion). Inspired by the shape and recovery process of a Rubik’s Cube, RCAFusion establishes an information interaction pathway among different modalities, and it achieves a more comprehensive information crossover through the simultaneous spatial attention, channel attention, and self-attention mechanisms, which enhance the feature extraction effect in the fusion architecture. Experimental results demonstrate that RCAFusion outperforms mainstream fusion algorithms in several metrics and obtains the highest score in the objective fused image metric Qabf. Moreover, the fused images output by RCAFusion have good results in the image object detection task and can achieve 95.4% mAP using the yolov8m model in the MSRS open source datasets. Pre-trained model and code are available at https://github.com/vehicle-AngLi/RCAFusion.