Generating Interaction Behavior During a Dyadic Conversation Using a Diffusion Model

Automatic interaction motion generation is crucial for developing autonomous conversation systems. In a conversation situation, the behavior and voice of the conversation partner should be considered for natural motion generation, for example, nodding at appropriate times. However, current data-driven motion generation models do not consider the interaction factor because they generate motion from audio or text modalities. In this study, an interaction diffusion model (IDM) is proposed to generate interaction behaviors based on a diffusion approach and masking features. All participants in the conversation are simultaneously considered in the IDM, and a mask for ignoring some features is applied to conditional generation. The proposed IDM was applied to the motion generation task of an expert's gesture, and the nature of the generated results was investigated. The results suggest that the IDM can generate expert-like motion within approximately 30ms.