Sempart: Self-supervised Multi-resolution Partitioning of Image Semantics

Accurately determining salient regions of an image is challenging when labeled data is scarce. DINO-based self-supervised approaches have recently leveraged meaningful image semantics captured by patch-wise features for locating foreground objects. Recent methods have also incorporated intuitive priors and demonstrated value in unsupervised methods for object partitioning. In this paper, we propose Sempart, which jointly infers coarse and fine bi-partitions over an image’s DINO-based semantic graph. Furthermore, Sempart preserves fine boundary details using graph-driven regularization and successfully distills the coarse mask semantics into the fine mask. Our salient object detection and single object localization findings suggest that Sempart produces high-quality masks rapidly without additional post-processing and benefits from co-optimizing the coarse and fine branches.

1. Introduction

Identifying salient regions of an image prone to holding visual attention remains a long-standing fuzzy problem [59] relying significantly on carefully annotated data [51], [5], [54]. Recently self-supervised (SSL) mechanisms based on large-scale pre-trained backbones [9], [6], [22], such as DINO [7], have demonstrated increased capability in segmenting images [21], [30] and extracting objects in the foreground [41], [39], [54], [4], [42].