A RGB-Thermal Image Segmentation Method Based on Parameter Sharing and Attention Fusion for Safe Autonomous Driving

In this paper, we propose a new RGB-thermal image segmentation method based on parameter sharing and attention fusion for safe autonomous driving. An encoder-decoder network structure is adopted. The encoder, which has shared convolution layer parameters and private batch normalization layer parameters (parameter sharing scheme), is used to extract features from RGB and thermal images. The extracted features are then fused by spatial and channel attention. The output of each residual block is fused, and the self-learning weight is used to integrate the fusion information of all residual blocks of the same levels. Subsequently, the fused features are integrated through a feature integration (FI) module in the decoder. Cross-entropy supervision of segmentation and edge is performed on the outputs of the decoders. Our proposed method is evaluated and compared with 17 state-of-the-art image segmentation methods, both qualitatively and quantitatively on the MFNet dataset which includes various objects in urban scenes. The results show that the proposed method outperforms previous methods by at least 0.3% and 1.8% in MRecall and MIoU, respectively, providing foundations for the development of autonomous driving technologies for safety enhancement.

I. Introduction

As an important computer vision technique, image segmentation has been widely used in various driving vision tasks for safety improvement of autonomous vehicles [1], [2]. Reliable autonomous vehicle operations, especially in realistic urban environments, are highly dependent on effective and efficient image segmentation [3], [4], [5].