Hybrid Network Based on Hierarchical Multipatch Feature Encoder for Infrared Small Target Detection

With recent advances in artificial intelligence, deep learning networks with various structures are being applied to target detection. However, in the infrared small target detection (IRSTD) field, an appropriate network structure is still required to identify blurry and low-contrast targets accurately. Conventional U-net algorithms use skip connection and attention module because information loss occurs as the convolution layer becomes deeper. However, the problem of not being able to recognize the entire image due to the limitations of convolution is fatal in IRSTD. To overcome these limitations, we design an encoder that divides the image into multiple small pieces and stacks them hierarchically to extract features. Therefore, the proposed network has a hybrid encoder structure that combines a convolution-based multiscale encoder to extract local information and a hierarchical-based multipatch encoder to extract global information by running in parallel. To effectively fuse the hierarchical information obtained from each layer, the loss function, which plays an important role in learning, has also been changed to suit the hybrid encoder. The proposed algorithm, named hierarchical multipatch feature and attention multiscale feature fusion U-net (HMAMFU-net), can guarantee effective target detection performance with two datasets: NUDT-SIRST and NUAA-SIRST, and performance analysis is conducted with other state-of-the-art algorithms. PyTorch implementation is available at
https://github.com/skylih87/HMAMFU-net
.