Estimating the Generation Timing of Responsive Utterances by Active Listeners of Spoken Narratives

Narrating is a fundamental human desire. In order to satisfy the desire, it is more effective to have multiple listeners listen to narratives than to have one listener does. Conversational agents, e.g., communication robots and smart speakers, are expected to listen to narratives, instead of human listeners. For such agents to be recognized as narrative listeners, it is necessary to generate responsive utterances at appropriate timings. This paper proposes a method to estimate the generation timings of responsive utterances by active listeners of spoken narratives. The proposed method estimates them by encoding acoustic and linguistic information extracted from narratives with a transformer-based approach. To evaluate the performance of the proposed method, an experiment was conducted using 131,616 responsive utterances to narratives. The experimental results demonstrated that the proposed method was effective for the estimation and both the acoustic and linguistic information was useful.