Light Transport Induced Domain Adaptation for Semantic Segmentation in Thermal Infrared Urban Scenes

Semantic segmentation in urban scenes is widely used in applications of intelligent transportation systems (ITS). In urban scenes, thermal infrared (TIR) images can be captured in weak illumination conditions or in the presence of obscuration (e.g., light fog, smoke). Therefore, TIR images have great potential to endow automated intelligent vehicles or assist navigation systems. However, TIR imaging is blurry and low-contrast due to the absorption by atmospheric gases and heat transfer effect. Hence, TIR semantic segmentation in urban scenes has rarely been explored even though it has a wide range of scenarios in ITS. To overcome this limitation, we analyze the light transport of TIR light. Our analysis reveals that contours are the reliable features shared by TIR and Visible Spectrum (VS) light. Inspired by this, we attempt to transfer joint features from VS domain to TIR domain. Thus, we propose a curriculum domain adaptation method to guide the TIR urban scene semantic segmentation task from VS domain through contours. Moreover, to evaluate the proposed model, we build TIR-SS: an open-for-request dataset consisting of TIR images and pixel level annotations of 8 classes in urban scenes. Qualitative and quantitative experimental results on the dataset indicate that the proposed domain adaptation method outperforms related methods on this TIR semantic segmentation task.

I. Introduction

Semantic segmentation in urban scenes has been widely applied in intelligent transportation systems (ITS) [1]–[3]. However, diverse illumination and weather conditions affect the performance of semantic segmentation. Thermal infrared (TIR) imagery is able to detect objects with a temperature above absolute zero. Based on this property, TIR imaging is functional in weak illumination and critical weather conditions (e.g., light fog, smoke). Thus, it has been widely used in applications for pedestrians [4]–[6]. In addition, TIR imagery is applied in ophthalmology [7], surgery [7], biometric analysis [8], military object detection and many other fields [14] based on the temperature sensitivity and non-invasive, non-contact detection capability. In recent years, the advance of sensor technology has led to a drop of the price and a growing application range of TIR imaging [14], [15].