Human Prediction for the Natural Instruction of Handovers in Human Robot Collaboration

Human robot collaboration is aspiring to establish hybrid work environments in accordance with specific strengths of humans and robots. We present an approach of flexibly integrating robotic handover assistance into collaborative assembly tasks through the use of natural communication. For flexibly instructed handovers, we implement recent Convolutional Neural Networks in terms of object detection and grasping of arbitrary objects based on an RGB-D camera equipped to a robot following the eye-in-hand principle. In order to increase fluency and efficiency of the overall assembly process, we investigate the human ability to instruct the robot predictively with voice commands. We conduct a user study quantitatively and qualitatively evaluating the predictive instruction in order to achieve just-in-time handovers of tools needed for following subtasks. We compare our predictive strategy with a pure manual assembly having all tools in direct reach and a step-by-step reactive handover. The results reveal that the human is able to predict the handover comparable to algorithm-based predictors. Nevertheless, human prediction does not rely on extensive prior knowledge and is thus suitable for more flexible usage. However, the cognitive workload for the worker is increased compared to manual or reactive assembly.