Fusion of Dilated Convolution in CNN and Transformer Networks for Hyperspectral Image Classification

Although existing Convolutional Neural Networks (CNNs) have exhibited commendable performance in hyperspectral image classification, they often emphasize local features. In recent years, transformers have garnered interest for capturing global features in hyperspectral images. This paper introduces a hyperspectral image classification framework, Dilated Convolution in CNN and Transformer Networks (DCCTnet), which employs various types of convolutions for local feature extraction and integrates dilated convolutions into multi-head self-attention mechanisms. Initially, multi-stage dilated convolutions are applied for multi-level feature extraction from the image. Following this, a grouped convolution is employed for feature fusion. To integrate CNN and transformers seamlessly, a pioneering Multi-Scale Multi-Head Self-Attention mechanism (MS-MHSA) is proposed, incorporating multiple dilated convolutions in MHSA to capture local-global multi-scale hyperspectral features. This mechanism is seamlessly integrated with the CNN branch, harnessing the strengths of both CNN and transformers. Through extensive experiments on two standard datasets, our proposed method demonstrates higher classification accuracy compared to other state-of-the-art networks.