An Architecture for Emotional Facial Expressions as Social Signals | IEEE Journals & Magazine | IEEE Xplore

An Architecture for Emotional Facial Expressions as Social Signals


Abstract:

We focus on affective architecture issues relating to the generation of expressive facial behaviour, critique approaches that treat expressive behaviour as only a mirror ...Show More

Abstract:

We focus on affective architecture issues relating to the generation of expressive facial behaviour, critique approaches that treat expressive behaviour as only a mirror of internal state rather than as also a social signal and discuss the advantages of combining the two approaches. Using the FAtiMA architecture, we analyse the requirements for generating expressive behavior as social signals at both reactive and cognitive levels. We discuss how facial expressions can be generated in a dynamic fashion. We propose generic architectural mechanisms to meet these requirements based on an explicit mind-body loop and Theory of Mind (ToM) processing. A illustrative scenario is given.
Published in: IEEE Transactions on Affective Computing ( Volume: 12, Issue: 2, 01 April-June 2021)
Page(s): 293 - 305
Date of Publication: 19 March 2019

ISSN Information:

Funding Agency:

References is not available for this document.

1 Introduction

This paper poses the problem of how to incorporate a generative account of expressive behaviour into an affective architecture, focusing on facial expressions. Expressive behaviour using the body posture, gesture, glance, facial expression is an significant component of communicative content alongside the verbal channel, and is therefore required for social agents, whether robots or graphical characters. Facial expressions are considered particularly important for agents that have a face (some robots do not), since this is often the focus of glance by an interaction partner. With more than forty muscle groups [1], the face has a wide range of movements and thus substantial expressivity. It has been argued that more than half of expressive behaviour relates to facial expressions [2].

Select All
1.
P. Ekman and W. V. Friesen, Facial action coding system: Investigators guide, Consulting Psychologists Press, 1978.
2.
A. Mehrabian, Nonverbal Communication, Evanston, IL, USA:Routledge, 2017.
3.
M. F. Jung, "Affective Grounding in Human-Robot Interaction", Proc. ACM/IEEE Int. Conf. Human-Robot Interaction, pp. 263-273, 2017.
4.
P. M. Cole, "Children’s spontaneous control of facial expression", Child Develop., vol. 57, pp. 1309-1321, 1986.
5.
C. Breazeal, "Role of expressive behaviour for robots that learn from people", Philosophical Trans. Roy. Soc. B: Biological Sci., vol. 364, pp. 3527-3538, 2009.
6.
T. W. Bickmore, "Relational agents: Effecting change through human-computer relationships", 2003.
7.
L. Hall, C. Hume, S. Tazzyman, A. Deshmukh, S. Janarthanam, H. Hastie, et al., "Map reading with an empathic robot tutor", Proc. 11th ACM/IEEE Int Conf. Human-Robot Interaction, pp. 567-567, 2016.
8.
R. Gockley, A. Bruce, J. Forlizzi, M. Michalowski, A. Mundell, S. Rosenthal, et al., "Designing robots for long-term social interaction", Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., pp. 1338-1343, 2005.
9.
P. Ekman and W. V. Friesen, Unmasking the face: A guide to recognizing emotions from facial clues, Los Altos, CA:Malor Books, 2003.
10.
D. Premack and G. Woodruff, "Does the chimpanzee have a theory of mind?", Behavioral Brain Sci., vol. 1, no. 4, pp. 515-526, 1978.
11.
C. M. D. Melo, P. Carnevale and J. Gratch, "The Effect of Expression of Anger and Happiness in Computer Agents on Negotiations with Humans", Proc. 10th Int. Conf. Auton. Agents Multiagent Syst., pp. 937-944, 2011.
12.
R. Hoegen, G. Stratou and J. Gratch, "Incorporating emotion perception into opponent modeling for social dilemmas", Proc. 16th Conf. Auton. Agents MultiAgent Syst., pp. 801-809, 2017.
13.
Z. Ambadar, J. F. Cohn and L. I. Reed, "All smiles are not created equal: Morphology and timing of smiles perceived as amused polite and embarrassed/nervous", J. Nonverbal Behavior, vol. 33, pp. 17-34, 2009.
14.
P. M. Niedenthal, M. Mermillod, M. Maringer and U. Hess, "The Simulation of Smiles (SIMS) model: Embodied simulation and the meaning of facial expression", Behavioral Brain Sci., vol. 33, pp. 417-433, 2010.
15.
P. Ekman and W. V. Friesen, "", Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues, 2003.
16.
C. Ritter and R. Aylett, "The partial poker-face", Proc. Int. Conf. Intell. Virtual Agents, pp. 479-482, 2015.
17.
J. Gratch, Z. Nazari and E. Johnson, "The Misrepresentation Game: How to win at negotiation while seeming like a nice guy", Proc. Int. Conf. Auton. Agents Multiagent Syst., pp. 728-737, 2016.
18.
B. M. DePaulo, S. E. Kirkendol, D. A. Kashy, M. M. Wyer and J. A. Epstein, "Lying in everyday life", J. Personality Soc. Psychology, vol. 70, pp. 979-995, 1996.
19.
A. J. Fridlund, Human Facial Expression, New York, NY, USA:Academic, 1994.
20.
P. Ekman, "Should we call it expression or communication?", Innovation: Eur. J. Soc. Sci. Res., vol. 10, no. 4, pp. 333-344, 1997.
21.
G. Horstmann, "What do facial expressions convey: Feeling states behavioral intentions or action requests?", Emotion, vol. 3, pp. 150-166, 2003.
22.
A. Scarantino, "Do emotions cause actions and if so how?", Emotion Rev., vol. 9, no. 4, pp. 326-334, 2017.
23.
L. R. Goldberg, "An Alternative ”Description of personality”: The big-five factor structure", J. Personality Soc. Psychology, vol. 59, pp. 1216-1229, 1990.
24.
A. Ortony, G. L. Clore and A. Collins, The Cognitive Structure of Emotiions, Cambridge, U.K.:Cambridge Univ. Press, 1988.
25.
A. S. Manstead and A. H. Fischer, "Social appraisal: The social world as object of and influence on appraisal processes" in Appraisal Processes in Emotion: Theory Methods Research, London, U.K.:Oxford Univ. Press, 2001.
26.
B. Parkinson, A. H. Fischer and A. S. Manstead, Emotion in Social Relations: Cultural Group and Interpersonal Processes, Evanston, IL, USA:Routledge, 2004.
27.
R. S. Lazarus and S. Folkman, "Transactional theory and research on emotions and coping", Eur. J. Personality, vol. 1, no. 3, pp. 141-169, 1987.
28.
J. Dias and A. Paiva, "Feeling and reasoning: A computational model for emotional characters", Proc. Portuguese Conf. Artif. Intell., pp. 127-140, 2005.
29.
J. Dias, R. Aylett, A. Paiva and H. Reis, "The great deceivers: Virtual agents and believable lies", Proc. Annu. Meeting Cognitive Sci. Soc., vol. 35, no. 35, pp. 2189-2194, 2013.
30.
S. Mascarenhas, J. Dias, R. Prada and A. Paiva, "A dimensional model for cultural behavior in virtual agents", Appl. Artif. Intell., vol. 24, no. 6, pp. 552-574, 2010.

References

References is not available for this document.