How Much Progress Did I Make? An Unexplored Human Feedback Signal for Teaching Robots

Enhancing the expressiveness of human teaching is vital for both improving robots’ learning from humans and the human-teaching-robot experience. In this work, we characterize and test a little-used teaching signal: progress, designed to represent the completion percentage of a task. We conducted two online studies with 76 crowd-sourced participants and one public space study with 40 non-expert participants to validate the capability of this progress signal. We find that progress indicates whether the task is successfully performed, reflects the degree of task completion, identifies unproductive but harmless behaviors, and is likely to be more consistent across participants. Furthermore, our results show that giving progress does not require extra workload and time. An additional contribution of our work is a dataset of 40 non-expert demonstrations from the public space study through an ice cream topping-adding task, which we observe to be multi-policy and sub-optimal, with sub-optimality not only from teleoperation errors but also from exploratory actions and attempts. The dataset is available at https://github.com/TeachingwithProgress/Non-Expert_Demonstrations.

I. Introduction

Robots have already firmly become part of our daily lives, making it crucial to learn from users, especially non-expert users. Learning from Demonstration (LfD) enables robots to learn new skills by observing expert policies [1], [2] while Learning from Human Feedback (LfHF) allows robots to adapt to human preferences or correct wrong behaviors by learning or shaping a policy [3], [4], [5]. More recent work has further shown that using human feedback and demonstrations together can make learning even more effective by reducing the data needs for human feedback [6] and loosening the requirements of demonstrations to be near-optimal [7]. However, while interest in learning fully or partially from humans is high, there is relatively little research on what the most effective forms of human feedback are, especially in combination with human demonstrations.