Visual Task Recognition for Human-Robot Teams

Human teammates in human-robot teams operate in uncertain, dynamic environments to accomplish a wide range of tasks. These tasks often involve multiple activity components: gross motor, fine-grained motor, tactile, cognitive, visual, speech and auditory. Most existing task recognition algorithms focus primarily on detecting tasks involving gross and fine-grained motor components; however, some tasks (e.g., assessing a victim’s triage level) may involve little to no motor components. Robots need a holistic understanding of a task’s various activity components in order to be aware of the human’s current task state. The presented algorithm detects the tasks’ visual activity component for a human-robot team operating in a non-sedentary supervisory environment. Metrics acquired from a wearable eye tracker and head motion tracker are used to train the machine learning-based visual task recognition algorithm.