An Architecture for Emotional Facial Expressions as Social Signals | IEEE Journals & Magazine | IEEE Xplore

An Architecture for Emotional Facial Expressions as Social Signals


Abstract:

We focus on affective architecture issues relating to the generation of expressive facial behaviour, critique approaches that treat expressive behaviour as only a mirror ...Show More

Abstract:

We focus on affective architecture issues relating to the generation of expressive facial behaviour, critique approaches that treat expressive behaviour as only a mirror of internal state rather than as also a social signal and discuss the advantages of combining the two approaches. Using the FAtiMA architecture, we analyse the requirements for generating expressive behavior as social signals at both reactive and cognitive levels. We discuss how facial expressions can be generated in a dynamic fashion. We propose generic architectural mechanisms to meet these requirements based on an explicit mind-body loop and Theory of Mind (ToM) processing. A illustrative scenario is given.
Published in: IEEE Transactions on Affective Computing ( Volume: 12, Issue: 2, 01 April-June 2021)
Page(s): 293 - 305
Date of Publication: 19 March 2019

ISSN Information:

Funding Agency:

Author image of Ruth Aylett
MACS, Heriot-Watt University, Edinburgh, United Kingdom
Ruth Aylett became a professor of Computer Science at Heriot-Watt University in 2004 where she is a member of the Edinburgh Centre for Robotics and researches social agents, human-robot interaction and affective computing. She currently leads a UKRC-funded project SoCoRo investigated the development of a robot trainer in social signal recognition for high-functioning adults with an ASD.
Ruth Aylett became a professor of Computer Science at Heriot-Watt University in 2004 where she is a member of the Edinburgh Centre for Robotics and researches social agents, human-robot interaction and affective computing. She currently leads a UKRC-funded project SoCoRo investigated the development of a robot trainer in social signal recognition for high-functioning adults with an ASD.View more
Author image of Christopher Ritter
CITEC, Bieldefeld University, Bieldefeld, Germany
Christopher Ritter received the German diploma in computer sciences from Friedrich-Alexander University Erlangen-Nuremberg. He is currently working toward the doctoral degree in computer sciences at the University of Bielefeld, Social Cognitive Systems group, CITEC.
Christopher Ritter received the German diploma in computer sciences from Friedrich-Alexander University Erlangen-Nuremberg. He is currently working toward the doctoral degree in computer sciences at the University of Bielefeld, Social Cognitive Systems group, CITEC.View more
Author image of Mei Yii Lim
School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, United Kingdom
Mei Yii Lim received the PhD degree from Heriot-Watt University, in 2007. She has worked as a post-doc researcher on EU-funded projects eCIRCUS, eCUTE, SOCIETIES, LIREC and EMOTE. She is currently a researcher on the UK-funded project SoCoRo.
Mei Yii Lim received the PhD degree from Heriot-Watt University, in 2007. She has worked as a post-doc researcher on EU-funded projects eCIRCUS, eCUTE, SOCIETIES, LIREC and EMOTE. She is currently a researcher on the UK-funded project SoCoRo.View more
Author image of Frank Broz
School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, United Kingdom
Frank Broz received the PhD degree in robotics from the Carnegie-Mellon University Robotics Institute. He is an assistant professor of computer science with Heriot-Watt University. He was a senior research fellow with Plymouth University working on the Robot-ERA project before joining Heriot-Watt University in 2015 and researches artificial intelligence, human-robot interaction and social robotics.
Frank Broz received the PhD degree in robotics from the Carnegie-Mellon University Robotics Institute. He is an assistant professor of computer science with Heriot-Watt University. He was a senior research fellow with Plymouth University working on the Robot-ERA project before joining Heriot-Watt University in 2015 and researches artificial intelligence, human-robot interaction and social robotics.View more
Author image of Peter E McKenna
Department of Psychology, Heriot-Watt University, Edinburgh, United Kingdom
Peter E McKenna received the doctorate degree in psychology from Heriot-Watt University, Edinburgh. He is currently a research associate on the EPSRC funded SoCoRo project - he and the team are developing a socially competent robot to teach adults with an ASD social and employment skills.
Peter E McKenna received the doctorate degree in psychology from Heriot-Watt University, Edinburgh. He is currently a research associate on the EPSRC funded SoCoRo project - he and the team are developing a socially competent robot to teach adults with an ASD social and employment skills.View more
Author image of Ingo Keller
School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, United Kingdom
Ingo Keller received the diploma in computer science from Technische Universitt Dresden (TUD), Germany, in 2010. He joined Heriot-Watt University in 2014 to pursue the PhD degree and is researching interactive object learning in the area of Teachable Robots. He is also investigating aspects of gesture synthesis in the SoCoRo project.
Ingo Keller received the diploma in computer science from Technische Universitt Dresden (TUD), Germany, in 2010. He joined Heriot-Watt University in 2014 to pursue the PhD degree and is researching interactive object learning in the area of Teachable Robots. He is also investigating aspects of gesture synthesis in the SoCoRo project.View more
Author image of Gnanathusharan Rajendran
Department of Psychology, Heriot-Watt University, Edinburgh, United Kingdom
Gnanathusharan Rajendran received the PhD degree in developmental psychology from the University of Nottingham. He is an associate professor in psychology with Heriot-Watt University. He joined Heriot-Watt University in 2012 as a reader and specialises in typical and atypical cognitive and social development, digital education, and social robotics.
Gnanathusharan Rajendran received the PhD degree in developmental psychology from the University of Nottingham. He is an associate professor in psychology with Heriot-Watt University. He joined Heriot-Watt University in 2012 as a reader and specialises in typical and atypical cognitive and social development, digital education, and social robotics.View more

1 Introduction

This paper poses the problem of how to incorporate a generative account of expressive behaviour into an affective architecture, focusing on facial expressions. Expressive behaviour using the body posture, gesture, glance, facial expression is an significant component of communicative content alongside the verbal channel, and is therefore required for social agents, whether robots or graphical characters. Facial expressions are considered particularly important for agents that have a face (some robots do not), since this is often the focus of glance by an interaction partner. With more than forty muscle groups [1], the face has a wide range of movements and thus substantial expressivity. It has been argued that more than half of expressive behaviour relates to facial expressions [2].

Author image of Ruth Aylett
MACS, Heriot-Watt University, Edinburgh, United Kingdom
Ruth Aylett became a professor of Computer Science at Heriot-Watt University in 2004 where she is a member of the Edinburgh Centre for Robotics and researches social agents, human-robot interaction and affective computing. She currently leads a UKRC-funded project SoCoRo investigated the development of a robot trainer in social signal recognition for high-functioning adults with an ASD.
Ruth Aylett became a professor of Computer Science at Heriot-Watt University in 2004 where she is a member of the Edinburgh Centre for Robotics and researches social agents, human-robot interaction and affective computing. She currently leads a UKRC-funded project SoCoRo investigated the development of a robot trainer in social signal recognition for high-functioning adults with an ASD.View more
Author image of Christopher Ritter
CITEC, Bieldefeld University, Bieldefeld, Germany
Christopher Ritter received the German diploma in computer sciences from Friedrich-Alexander University Erlangen-Nuremberg. He is currently working toward the doctoral degree in computer sciences at the University of Bielefeld, Social Cognitive Systems group, CITEC.
Christopher Ritter received the German diploma in computer sciences from Friedrich-Alexander University Erlangen-Nuremberg. He is currently working toward the doctoral degree in computer sciences at the University of Bielefeld, Social Cognitive Systems group, CITEC.View more
Author image of Mei Yii Lim
School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, United Kingdom
Mei Yii Lim received the PhD degree from Heriot-Watt University, in 2007. She has worked as a post-doc researcher on EU-funded projects eCIRCUS, eCUTE, SOCIETIES, LIREC and EMOTE. She is currently a researcher on the UK-funded project SoCoRo.
Mei Yii Lim received the PhD degree from Heriot-Watt University, in 2007. She has worked as a post-doc researcher on EU-funded projects eCIRCUS, eCUTE, SOCIETIES, LIREC and EMOTE. She is currently a researcher on the UK-funded project SoCoRo.View more
Author image of Frank Broz
School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, United Kingdom
Frank Broz received the PhD degree in robotics from the Carnegie-Mellon University Robotics Institute. He is an assistant professor of computer science with Heriot-Watt University. He was a senior research fellow with Plymouth University working on the Robot-ERA project before joining Heriot-Watt University in 2015 and researches artificial intelligence, human-robot interaction and social robotics.
Frank Broz received the PhD degree in robotics from the Carnegie-Mellon University Robotics Institute. He is an assistant professor of computer science with Heriot-Watt University. He was a senior research fellow with Plymouth University working on the Robot-ERA project before joining Heriot-Watt University in 2015 and researches artificial intelligence, human-robot interaction and social robotics.View more
Author image of Peter E McKenna
Department of Psychology, Heriot-Watt University, Edinburgh, United Kingdom
Peter E McKenna received the doctorate degree in psychology from Heriot-Watt University, Edinburgh. He is currently a research associate on the EPSRC funded SoCoRo project - he and the team are developing a socially competent robot to teach adults with an ASD social and employment skills.
Peter E McKenna received the doctorate degree in psychology from Heriot-Watt University, Edinburgh. He is currently a research associate on the EPSRC funded SoCoRo project - he and the team are developing a socially competent robot to teach adults with an ASD social and employment skills.View more
Author image of Ingo Keller
School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, United Kingdom
Ingo Keller received the diploma in computer science from Technische Universitt Dresden (TUD), Germany, in 2010. He joined Heriot-Watt University in 2014 to pursue the PhD degree and is researching interactive object learning in the area of Teachable Robots. He is also investigating aspects of gesture synthesis in the SoCoRo project.
Ingo Keller received the diploma in computer science from Technische Universitt Dresden (TUD), Germany, in 2010. He joined Heriot-Watt University in 2014 to pursue the PhD degree and is researching interactive object learning in the area of Teachable Robots. He is also investigating aspects of gesture synthesis in the SoCoRo project.View more
Author image of Gnanathusharan Rajendran
Department of Psychology, Heriot-Watt University, Edinburgh, United Kingdom
Gnanathusharan Rajendran received the PhD degree in developmental psychology from the University of Nottingham. He is an associate professor in psychology with Heriot-Watt University. He joined Heriot-Watt University in 2012 as a reader and specialises in typical and atypical cognitive and social development, digital education, and social robotics.
Gnanathusharan Rajendran received the PhD degree in developmental psychology from the University of Nottingham. He is an associate professor in psychology with Heriot-Watt University. He joined Heriot-Watt University in 2012 as a reader and specialises in typical and atypical cognitive and social development, digital education, and social robotics.View more

References

References is not available for this document.