A web-based infrastructure for recording user demonstrations of mobile manipulation tasks

Learning from demonstration (LfD) is a common technique applied to many problems in robotics, such as populating grasp databases, training for reinforcement learning of high-level skill sets and bootstrapping motion planners. While such approaches are generally highly valued, they rely on the often time-consuming process of gathering user demonstrations, and hence it becomes difficult to attain a sizeable dataset. In this paper, we present a tool capable of recording large numbers of high-dimensional demonstrations of mobile manipulation tasks provided by non-experts in the field. Our tool accomplishes this via a web interface that requires no additional software to be installed beyond a web browser, as well as a scalable architecture that is capable of supporting 10 concurrent demonstrators on a single server. Our architecture employs a lightweight simulation environment to reduce unnecessary computations and improve performance. Furthermore, we show how our tool can be used to gather a large set of demonstrations of a mobile manipulation task by leveraging existing crowdsource platforms. The data set collected has been made available to the robotics community. We also present experiments in which we apply demonstrations collected through our infrastructure to teach a robot how to grasp, to teach a robot how to perform dexterous manipulation tasks such as scooping and to accelerate motion planning for full-body manipulation tasks.

I. Introduction

Developing robotic systems that learn from human demonstrations of everyday motions and tasks has proven to be a popular and useful approach in dealing with many problems in mobile manipulation over the last two decades [14], [5], [11], [10], [9], [17]. These techniques are valuable because many of the tasks normally performed by people are complex in structure, the components of which must be explicitly described for the robot. Furthermore, the collection of relevant demonstrations is an integral part of virtually all Learning from Demonstration (LfD) techniques.