A Dual-Branch Network Based on Transformer and Depthwise Convolution for Hyperspectral Image Classification

In the past few years, the application of convolutional neural networks (CNNs) has become prevalent in hyperspectral image (HSI) classification research. However, since the convolutional operation of the CNN is performed on the local space, it is difficult to deal with the spectral global dependency of HSIs. Moreover, because of the small perceptual field of the CNN, it is often necessary to design a very deep network, which limits its classification performance and classification efficiency. To tackle the mentioned challenges, this paper proposes a dual-branch network based on transformer and depthwise convolution for HSI classification (DBTD). DBTD employs a dual-branch end-to-end classification model based on transformer and depthwise convolution that adaptively extracts spectral features of HSIs using an attention mechanism on the main branch, and captures the long-distance dependency between features at a global scale. It uses the other depthwise convolutional branch to extract spatial neighborhood features that assists in classification. According to the experimental findings, the proposed approach outperforms the compared methods across various public datasets, demonstrating its feasibility and effectiveness in HSI classification.