Salient Object Detection with Recurrent Fully Convolutional Networks

Deep networks have been proved to encode high-level features with semantic meaning and delivered superior performance in salient object detection. In this paper, we take one step further by developing a new saliency detection method based on recurrent fully convolutional networks (RFCNs). Compared with existing deep network based methods, the proposed network is able to incorporate saliency prior knowledge for more accurate inference. In addition, the recurrent architecture enables our method to automatically learn to refine the saliency map by iteratively correcting its previous errors, yielding more reliable final predictions. To train such a network with numerous parameters, we propose a pre-training strategy using semantic segmentation data, which simultaneously leverages the strong supervision of segmentation tasks for effective training and enables the network to capture generic representations to characterize category-agnostic objects for saliency detection. Extensive experimental evaluations demonstrate that the proposed method compares favorably against state-of-the-art saliency detection approaches. Additional validations are also performed to study the impact of the recurrent architecture and pre-training strategy on both saliency detection and semantic segmentation, which provides important knowledge for network design and training in the future research.

1 Introduction

Saliency detection can be generally divided into two subcategories: salient object segmentation [1], [2], [3] and eye-fixation detection [4], [5]. This paper mainly focus on salient object segmentation, which aims to highlight the full extent of the most conspicuous object regions in images. It can be used as a pre-processing step to facilitate a wide range of vision applications, e.g., image segmentation [6], image retrieval [7], [8], visual tracking [9], etc., and have received increasingly more interests from the community. Although much progress has been made, it is still a very challenging task to develop effective algorithms capable of handling real world adverse scenarios.