MGC: MLP-Guided CNN Pretraining Using a Small-Scale Dataset for Remote Sensing Images

To overcome the inherent domain gap between natural images and remote sensing images (RSIs), it is highly desirable to develop pretraining methods specifically for RSIs. Considering the lack of widely recognized large-scale benchmarks like ImageNet in the RSI community and limited computational resources, this article proposes multilayer perceptron (MLP)-guided convolutional neural network (CNN) (MGC), a method that employs an MLP to guide the pretraining of a CNN from small-scale datasets for RSIs. MGC has two encoders, each consisting of a CNN branch and an MLP branch. We first contrast pairwise samples from the same type of branches or different types of branches across the encoders and employ a positive-pair guidance strategy to explore consistency. Due to the inherent locality issue of shallow layers in a CNN, the CNN branches often do not attend to correct foreground regions such as objects, regions of interest, and land coverage. Therefore, we further propose an attention guidance strategy to guide the CNN branches to focus on foreground regions and learn discriminative representations effectively. The proposed MGC method is validated by pretraining a CNN model using the MGC and applying it to different downstream tasks including scene classification, rotated object detection, semantic segmentation, and change detection on ten datasets. Results have confirmed the effectiveness of the proposed MGC. Our code will be released at: https://github.com/benesakitam/MGC.

I. Introduction

Deep learning has played a major role in remote sensing image (RSI) interpretation [1], [2], [3], [4], [5], [6], [7], [8]. Many convolutional neural network (CNN)-based models, such as ResNet-50 [9], often rely on pretraining the models on ImageNet [10]. The inherent domain gap in data characteristics and imaging mechanisms between natural images and RSIs limit the performance of these models. Although some works [11], [12], [13], [14] have explored pretraining specifically on RSIs, they require diverse sources of RSIs and there is still no commonly accepted pretraining benchmark like ImageNet in the RSI community. On the other hand, pretraining on a large-scale dataset requires significant computing resources, making it only feasible in a limited number of research institutions. This article aims to generate competitive pretrained models on randomly sampled small-scale RSIs and with limited computing resources, e.g., four NVIDIA V100 GPUs.