A Transformer-based Video Segmentation Method for Road Weather Conditions

Detecting road weather conditions is very important for autonomous driving systems. Many existing studies use image classification or segmentation methods to recognize road weather conditions, but one single image can be easily affected by environmental factors such as lighting, resulting in lower image quality and false detections. In this paper, we propose a video segmentation method to detect dry, wet and snowy road areas. By fusing the features of consecutive image frames in the video, the impact of poor quality images on the detection results can be reduced, improving the robustness of the algorithm. Meanwhile, we propose the Temporal Feature Fusion Block (TFFB) based on the Transformer decoder to achieve the feature-level fusion, avoiding the image registration required for traditional temporal image fusion. This reduces the algorithm complexity and improves real-time performance. In order to verify the effectiveness of the algorithm, we constructed a dataset of road weather conditions by collecting images and manually annotated them at the pixel level. The dataset has 12 video sequences with a total of about 5000 frames. Experimental results show that the mean Intersection over Union (mIoU) of road weather condition segmentation can be 87.62% by multiple frame feature fusion, which confirms the effectiveness of the algorithm.