Real-Time Speech Emotion Analysis for Smart Home Assistants

Artificial Intelligence (AI) based Speech Emotion Recognition (SER) has been widely used in the consumer field for control of smart home personal assistants, with many such devices on the market. However, with the increase in computational power, connectivity, and the need to enable people to live in the home for longer though the use of technology, then smart home assistants that could detect human emotion will improve the communication between a user and the assistant enabling the assistant of offer more productive feedback. Thus, the aim of this work is to analyze emotional states in speech and propose a suitable method considering performance verses complexity for deployment in Consumer Electronics home products, and to present a practical live demonstration of the research. In this article, a comprehensive approach has been introduced for the human speech-based emotion analysis. The 1-D convolutional neural network (CNN) has been implemented to learn and classify the emotions associated with human speech. The paper has been implemented on the standard datasets (emotion classification) Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) and Toronto Emotional Speech Set database (TESS) (Young and Old). The proposed approach gives 90.48%, 95.79% and 94.47% classification accuracies in the aforementioned datasets. We conclude that the 1-D CNN classification models used in speaker-independent experiments are highly effective in the automatic prediction of emotion and are ideal for deployment in smart home assistants to detect emotion.

I. Introduction

Speech Emotion Recognition (SER) was first proposed in 1997 by Picard [1] and has attracted widespread attention. It is well known that language communication is the preferred method when communicating with others in daily life, and human language is first formed through speech. It can be said that speech plays a decisive supporting role in language. Human speech not only contains important semantic information, but also implies rich emotional information [2]. The aim of SER is to obtain the emotional states of a user derived from their speech [3], thereby achieving harmonious communication between humans or between humans and machines, and in this article, we refer to a machine as a smart home assistant.