Specular Highlights Detection Using a U-Net Based Deep Learning Architecture

Different lighting conditions surrounding an object can cause specular reflections, resulting in specular highlights in the captured image. These can interfere with image processing algorithms, leading to false interpretation of results. Hence, applications that demand consistent object appearance require that such highlights are detected and localized for subsequent processing. Using a slightly modified U-Net architecture, we propose a semantic segmentation model to localize specular highlights. To evaluate the segmentation performance, the IoU metric is used. This measures how good the segmentation prediction matches the corresponding reference annotation by dividing their intersection by their union (IoU). The model achieved a frequency-weighted and mean IoU of 0.83 and 0.75 respectively when tested on a benchmark dataset. Furthermore, the proposed network was also trained and tested on a custom dataset, focusing on flash light reflections in an underground tunnel environment. For this custom dataset, the model achieved a frequency-weighted and mean IoU of 0.98 and 0.80 respectively.