SG-DLCT: Saliency-Guided Dual-Level Collaborative Transformer for Image Captioning

Improving the generation of feature representations for images has been a central focus in the field of image captioning. However, existing approaches, whether based on grid features or region features, only consider the features within individual regions without capturing the interactions between objects. In this paper, we propose a novel Saliency-Guided Dual-Level Collaborative Transformer (SG-DLCT), which leverages saliency information to guide the network in capturing the importance of different objects and their interrelationships. Concretely, we utilize a pre-trained RGBD salient object ranking model to obtain saliency information. Building upon this, we propose the Saliency Feature Enhanced Module (SFEM), which optimizes region features using saliency values as weights, thereby guiding the model to focus on image regions that are more meaningful for generating image captions. Additionally, we propose the Saliency-Guided Cross Attention (SGCA), which leverages saliency information to model the interaction between region features and grid features, allowing the final model's features to benefit from the advantages of both types of features. Experimental results validate the significance of saliency information in image captioning tasks and demonstrate the effectiveness of our proposed method.