Multi-Task Active Learning for Simultaneous Emotion Classification and Regression

Emotion recognition, which aims to identify an individual’s emotional state from the acquired physiological or body signals, is very important in affective computing. Emotions have two common representations: categorical, e.g., happy, sad, etc., and dimensional (continuous), e.g., valence, arousal and dominance. Training a good emotion classification or regression model usually requires a large number of labeled data. However, the labeling process is very difficult. As emotions are subtle and uncertain, it usually requires multiple assessors to label each emotional instance to obtain the groundtruth categorical label or dimensional values. In this paper, we propose a multi-task active learning (MTAL) framework to query the most useful samples for labeling, which enables the efficient training of an emotion classification model and multiple emotion regression models simultaneously. This is novel and challenging, as all previous research considered only emotion classification or regression alone, but not simultaneously. Experimental results on the IEMOCAP dataset demonstrated that MTAL outperformed random selection and several state-of-the-art single task active learning approaches, i.e., with the same number of labeled samples, MTAL can obtain better emotion classification and regression models simultaneously.