Recent developments in social signal processing

Social signal processing has the ambitious goal of bridging the social intelligence gap between computers and humans. Nowadays, computers are not only the new interaction partners of humans, but also a privileged interaction medium for social exchange between humans. Consequently, enhancing machine abilities to interpret and reproduce social signals is a crucial requirement for improving computer-mediated communication and interaction. Furthermore, automated analysis of such signals creates a host of new applications and improvements to existing applications. The study of social signals benefits a wide range of domains, including human-computer interaction, interaction design, entertainment technology, ambient intelligence, health-care, and psychology. This paper briefly introduces the field and surveys its latest developments.

I. Introduction

When science fiction writers imagined a future for smarter computer systems, they often envisioned all-knowing, immensely capable, and purposeful machines. In February 2011, IBM's Watson computer won the Jeopardy! contest, displaying how close machines are to being ‘all- knowing’, thanks to their ability to store and sift through enormous amounts of factual information. While we may philosophically argue that ‘knowing’ is not applicable here in the strict sense, we cannot deny that Watson surpasses humans in its knowledge of trivia. However, when it comes to social intelligence, computers have a lot of room for improvement. We all use computers, and we know that they are never responsive to our emotions, moods, or to any kind of social context. Even an ‘all-knowing’ Watson computer would be a very poor dinner companion, as it lacks the skills to decide when to speak, and what to say. The newly emerging field of social signal processing aims at providing computers with the means of analysing and adequately representing human social signals, which will allow them to adapt and properly function in social settings.