Color-Aware Self-Supervised Learning for Scene Classification and Segmentation of Remote Sensing Images

Recently, fully supervised deep learning has achieved excellent success in remote sensing (RS) scene classification and segmentation. However, supervised learning requires tremendous labels, which are difficult to obtain in the field of RS. Self-supervised contrastive methods alleviate this problem by learning impressive transferable representations invariant to different data augmentations, e.g. color jittering. Such invariance could be harmful to RS scene classification and segmentation, which is sensitive to color changes. Therefore, we introduce a color-aware self-supervised learning framework (ColorSelf) for RS scene classification and segmentation. Our model encourages to preserve color-aware information in learned representation to improve their transferability. Extensive experiments on two challenging RS datasets demonstrate the proposed ColorSelf brings a significant performance improvement in both RS scene classification and segmentation task.

1. INTRODUCTION

Scene classification and segmentation play increasingly significant roles in the remote sensing images (RSI) interpretation field, which provide basic information for many high-level applications [1], [2], [3], [4], [5], such as urban planning, disaster response and management, forest monitoring, agricultural management, etc. Due to high capacity and effective way to learn semantic features, supervised deep learning algorithms almost dominate the filed of scene classification and segmentation [6], [7], which rely on requires tremendous labels. Driven by the rapidly development of remote sensing devices and platforms, the RSI with different sources and resolutions are constantly generated. However, Only a some portion of RSI were labeled with high quality. To this end, leveraging unlabeled data to improve model performance has gained more attention in the remote sensing community.