Self-supervised Learning with Local Contrastive Loss for Detection and Semantic Segmentation

We present a self-supervised learning (SSL) method suitable for semi-global tasks such as object detection and semantic segmentation. We enforce local consistency between self-learned features that represent corresponding image locations of transformed versions of the same image, by minimizing a pixel-level local contrastive (LC) loss during training. LC-loss can be added to existing self-supervised learning methods with minimal overhead. We evaluate our SSL approach on two downstream tasks – object detection and semantic segmentation, using COCO, PASCAL VOC, and CityScapes datasets. Our method outperforms the existing state-of-the-art SSL approaches by 1.9% on COCO object detection, 1.4% on PASCAL VOC detection, and 0.6% on CityScapes segmentation.

1. Introduction

Self-supervised learning (SSL) approaches learn generic feature representations from data in the absence of any external supervision. These approaches often solve an in-stance discrimination pretext task in which multiple trans-formations of the same image are required to generate similar learned features. Recent SSL methods have shown remarkable promise in global tasks such as classifying images by training simple classifiers on the features learned via instance discrimination [1], [2], [4], [17], [18]. However, global feature-learning SSL approaches do not explicitly retain spatial information thus rendering them ill-suited for semi-global tasks such as object detection, and instance and semantic segmentation [37], [43].