A Novel Speech Emotion Recognition Method via Incomplete Sparse Least Square Regression

In this letter, we propose a novel speech emotion recognition method based on least square regression (LSR) model, in which a novel incomplete sparse LSR (ISLSR) model is proposed and utilized to characterize the linear relationship between speech features and the corresponding emotion labels. In training the ISLSR model, both labeled and unlabeled speech data sets are utilized, where the use of unlabeled data set aims to enhance the compatibility of the model such that it is well suitable for the out-of-sample speech data. Another novelty of ISLSR lies in the capability of dealing with feature selection. To evaluate the performance of the proposed method, we conduct experiments on two emotional speech databases. The experimental results on both databases demonstrate that the proposed method achieves better recognition performance in compared with several state-of-the-art methods.

I. Introduction

Speech emotion recognition has been a very active research topic in the pattern recognition field. A major goal of emotion recognition from speech is to classify the speech utterances into one of the predefined emotion categories, e.g., anger, joy, sadness, fear, disgust, boredom, neutral [1]. Overall, an automatic speech emotion recognition system can be divided into two major parts, i.e., speech feature extraction versus emotion classification [2]. The main task of the first part is to extract the speech features that are related with the emotions of the speakers, whereas the latter one is to determine the emotion categories based on the extracted speech features. During the last decades, many speech emotion recognition methods had been proposed in the literature [2], among which the regression based approaches had been very popular in recent years [4]. One of the most commonly used approaches of applying regression model to speech emotion recognition is the ordinary least square regression (LSR) model. This method aims to seek a transformation matrix , such that the difference between emotion label matrix and transformed speech feature matrix is minimal. The optimization problem can be formulated as:

$$\arg \min_{\bf C} \Vert {\bf L} - {\bf CD}\Vert _F^2.\eqno{\hbox{(1)}}$$