Color-Aware Self-Supervised Learning for Scene Classification and Segmentation of Remote Sensing Images

Recently, fully supervised deep learning has achieved excellent success in remote sensing (RS) scene classification and segmentation. However, supervised learning requires tremendous labels, which are difficult to obtain in the field of RS. Self-supervised contrastive methods alleviate this problem by learning impressive transferable representations invariant to different data augmentations, e.g. color jittering. Such invariance could be harmful to RS scene classification and segmentation, which is sensitive to color changes. Therefore, we introduce a color-aware self-supervised learning framework (ColorSelf) for RS scene classification and segmentation. Our model encourages to preserve color-aware information in learned representation to improve their transferability. Extensive experiments on two challenging RS datasets demonstrate the proposed ColorSelf brings a significant performance improvement in both RS scene classification and segmentation task.