ConvGRU-Based Multiscale Frequency Fusion Network for PAN-MS Joint Classification

As a hot research topic in remote sensing, effectively integrating the advantageous features of multispectral and panchromatic images is the main challenge for fusing these two remote sensing images. This article proposes a multiscale frequency fusion network based on ConvGRU. To address the underutilization of texture features, we extract multiscale bandpass and low-pass sub-bands representing texture and content features through Contourlet decomposition. Multiscale bandpass sub-bands contain more comprehensive and concentrated texture details. Then, by proposing a multiscale frequency feature extractor based on ConvGRU, we effectively integrate and enhance sub-bands of different scales and frequencies, fully utilizing the characteristics of multispectral and panchromatic images and scale transmission. With these enhanced sub-band features, we obtain more comprehensive scale-enhanced texture features. Simultaneously, content features are also preserved as dual-source image features. Moreover, to reduce redundancy between fused features and make more efficient use of the obtained enhanced features, we designed an Inver-band integrator (IBI) module. It can fuse enhanced features at different scales, improve the complementarity between features, and thus achieve effective fusion. Experimental results demonstrate the effectiveness and robustness of our model on multiple datasets. Our codes are available at https://github.com/Xidian-AIGroup190726/GMFnet.