Bimodal-based Object Detection and Instance Segmentation Models for Substation Equipments

Detection and segmentation of the substation equipments is the important first step towards establishing an AI-based thermal fault detection of substation equipments. The traditional detection and segmentation methods have been built up based on the single mode of thermal or visible light image. In this paper, we propose the framework of bimodal fusion: the visible-light images and the temperature map, to establish the deep neural network models for object detection and instance segmentation of the substation equipments, based on the Mask R-CNN. In our private fused dataset, we realize and compare diverse fusion methods, including the pixel-based fusion, feature-based fusion and decision-level fusion methods for the detection and segmentation task of substation equipments. The comparison experiments shown that the FPN feature layer fusion model in the feature-based fusion can achieve better detection and segmentation effects than the others and the models of the single mode. We also demonstrate that the fused method can slightly improve the performance in the night scene by simulation. However, the improvement of performance measured by the mAP and AR of these method are all slight.