A Domain-aware Language-supervised Method for Image Emotion Classification

Image emotion classification (IEC) is designed to predict the main categories of emotional tendencies when people look at an affective image. Unlike common image classification tasks, IEC suffers from conceptual abstraction and high annotation costs. The language-supervised methods, i.e., SimEmotion, are designed to address the abstract nature of emotion. However, it is still trained individually on a specific dataset given a particular classifier and cannot effectively use valuable annotated data from other source domains, which limits training. We propose a domain-aware language-supervised image emotion classification prompt learning method, DaLs. Compared to current language-supervised methods, DaLs can employ fewer parameters. Moreover, in addition to aggregating binary-category datasets with the same category labels, our approach can also fuse datasets with different emotion models for effective multi-category classification experiments. Evaluations of four widely-used affective datasets, demonstrate that the proposed algorithm outperforms the state-of-the-art methods on IEC tasks.