An automated speech-language therapy tool with interactive virtual agent and peer-to-peer feedback

Widespread use of technology has brought about revolutionary changes in health domain. Treatment of speech and language disorders is one such field where computer-based techniques have the potential to provide readily available therapy at low cost by reducing the reliance on therapists and clinicians. Although significant research has been performed on computer-aided speech and language therapy, existing approaches do not focus on the nonverbal behavioral cues of a patient in a natural conversational scenario. In this paper, we propose an automated speech and language therapy tool with an intelligent and interactive virtual agent playing the role of a therapist. Besides giving feedback to the patient's response on various therapeutic tasks, the virtual therapist has the ability to carry out a conversation with the patient and give feedback on their nonverbal behavioral cues to help them improve their communicating ability. Additionally, our proposed system supports computerized peer-to-peer conversation in real-life scenarios which is also underrepresented in existing research concerning computer-assisted speech language therapy.