Ambiguous command understanding with commonsense

This paper proposes our approach to disambiguate human pointing gesture by using commonsense knowledge. Pointing gesture is a natural way to specify the object that a user needs in the “Bring something” service. However, error in gesture recognition makes robot difficult to select the correct candidate object when there are multiple objects located closely together. To solve this problem, Robot Technology (RT) Ontology was used to make assumption about human activity and necessary objects for the activity. By combining this commonsense reasoning with the spatial constrain from human's pointing direction, robot is able to find appropriate object. Experiment showed the successful rate of object selection was improved, and therefore robotic service becomes more intuitive.

I. Introduction

Identifying target object based on hand pointing gesture is a natural method for human-robot interface, however, it is subject to high level of uncertainty when applied to real life scenarios. As shown in Fig. 1, the uncertainty may comes from the angular error of pointing direction estimation, or from the overlap between objects. Because the problem is related to the arrangement of objects in real life, it is difficult to solve only by improving the gesture recognition method.