Saliency Detection via Recurrently Aggregating and Refining Cross-layer Residual Features

In the past few years, deep convolutional neural networks (CNNs) have benefited great progress in salient object detection due to the strong ability of feature representation. As different level of features can be reflected in different layers, traditional CNNs based methods usually integrate features from multiple layers to obtain appealing results. However, the fine details of salient objects tend to be lost because of multiple subsampling operations such as convolution and pooling. Meanwhile, the non-salient regions reflected in the low level features would be easily integrated into the final results incorrectly. In order to address these issues, we propose a novel deep saliency detection network by recurrently aggregating and refining cross-layer residual features (RARCRF). Instead of directly refining the intermediate saliency map as many previous plain network based methods do, we design a residual learning module and embed it into RARCRF to learn residual features. Then, the residual features learned from multiple layers are aggregated and refined step by step in a cross-layer manner. By this way, the complementary information in deep features extracted from different layers can be fully captured for detecting salient objects with different scales, i.e., the features integrated from low-level layers can serve to refine the details of detected salient objects while the features integrated from high-level layers with semantic information can benefit the locating of salient objects. Finally, different saliency detection results from different layers are fused to generate the final saliency map. Experimental results with ablation analysis on five benchmark datasets demonstrate that our proposed RARCRF outperforms other 14 state-of-the-art competitors.