Regularized Densely-Connected Pyramid Network for Salient Instance Segmentation

Much of the recent efforts on salient object detection (SOD) have been devoted to producing accurate saliency maps without being aware of their instance labels. To this end, we propose a new pipeline for end-to-end salient instance segmentation (SIS) that predicts a class-agnostic mask for each detected salient instance. To better use the rich feature hierarchies in deep networks and enhance the side predictions, we propose the regularized dense connections, which attentively promote informative features and suppress non-informative ones from all feature pyramids. A novel multi-level RoIAlign based decoder is introduced to adaptively aggregate multi-level features for better mask predictions. Such strategies can be well-encapsulated into the Mask R-CNN pipeline. Extensive experiments on popular benchmarks demonstrate that our design significantly outperforms existing state-of-the-art competitors by 6.3% (58.6% vs. 52.3%) in terms of the AP metric. The code is available at https://github.com/yuhuan-wu/RDPNet.

I. Introduction

As a fundamental image understanding technique, salient object detection (SOD) aims at segmenting the most eye-attracting objects in a natural image. Although recent SOD approaches [1]–[5] have achieved much success, their generated saliency maps cannot discriminate different salient instances, which has prevented many applications from applying SOD for instance-level image understanding [6]. Motivated by [7], in this paper, we tackle the more challenging case of SOD, called salient instance segmentation (SIS). SIS segments salient objects from an image and discriminates salient instances by associating each instance with a different label. SIS can facilitate more advanced tasks than SOD, such as image captioning [8], weakly-supervised semantic/instance segmentation [9], [10], and visual tracking [11].