Optimal Fusion-based Asymmetric Two-stream Networks for Multispectral Image Pedestrian Detection

As the fast developing of computer vision, pedestrian detection has become a hot topic of current research. Accurate and real-time detection of pedestrians from video sequences is of great practical importance in the field of intelligent video surveillance. Currently, most pedestrian detection systems are built on RGB images. However, RGB images are easily affected by light changes and weather conditions. In low-light scenes the quality is poor. Infrared information, however, relies only on the thermal radiation of the object and can clearly show the outline of the pedestrian. This paper therefore proposes a multispectral based pedestrian detection algorithm (multispectral images include RGB and IR images). Specifically, firstly, design an asymmetric two-stream network structure to extract RGB image and infrared image features respectively. Then a multimodal fusion mechanism is proposed to fuse RGB feature maps and IR feature maps for channel fusion. Experimental results demonstrate that our method improves the target detection properties. The map value reaches 0.955 on the dataset GTOT and the own proposed dataset School.