Global–Local 3-D Convolutional Transformer Network for Hyperspectral Image Classification

Benefiting from powerful feature extraction capabilities, convolutional neural networks (CNNs) have gained prominence in hyperspectral image (HSI) classification. Nevertheless, with restricted receptive fields of convolution kernels, CNN-based methods fail to learn complex characteristics of long-range sequences. Meanwhile, the vision transformer allows us to learn long-range dependencies in a global view, but local region features are ignored. To overcome these limitations, we propose a novel method entitled global–local 3-D convolutional transformer network (GTCT), where 3-D convolution is embedded in a dual-branch transformer to simultaneously capture global–local associations in both the spectral and spatial domains. In particular, the global–local spectral convolutional transformer (GECT) is designed to exploit global spectral sequence signatures and local spectral relationships between bands. Symmetrically, the global–local spatial convolutional transformer (GACT) is devised to exploit local spatial context features and global interactions among different pixels. In addition, multiscale global–local spectral–spatial information is adaptively fused with trainable weights by the weighted multiscale spectral–spatial feature interaction (WMSFI) module. It is worth noting that a spectral–spatial global attention mechanism (SSGAM) is incorporated into multihead convolutional attention to further integrate discriminative spectral–spatial information. Extensive experiments on four HSI datasets, including the GF-5 and ZY1-02D satellite HSIs, demonstrate the superiority of the proposed GTCT method over other state-of-the-art algorithms with fewer parameters and lower floating-point operations (FLOPs) in practical applications.