TFAD: An Image Multi-Label Recognition Method with Image-Text Powered Attention

The image multi-label classification task is a common and computationally intensive computer vision task. In this paper, we generate interpretable text features from images by visual large models, and then design a light-weight Text Fusion Attention Decoder (TFAD) framework by combining image and LLM enhanced text features, which extracts the important features by using the transformer structure and combines them with the average pooled features as well as the text class features by attention to achieve focusing on the features that are effective for classification and ignoring the effect of noise, which makes a better use of spatial data and achieves a higher accuracy as compared to the normal global average pooling. After extensive experiments with PASCAL-VOC, MS-COCO and WIDER-Attribute multi-label classification tasks, it is demonstrated that our method can achieve the accuracy improvement in the single-card scenario, which achieves a balance between computational effort and accuracy.