Efficient, Situated and Ontology based Referring Expression Generation for Human-Robot collaboration

In Human-Robot Interaction (HRI), ensuring nonambiguous communication between the robot and the human is a key point for carrying out fluently a collaborative task. With this work, we propose a method which allows the robot to generate the optimal set of assertions that are necessary in order to produce an unambiguous reference. In this paper, we present a novel approach to the Referring Expression Generation (REG) problem and its integration into a robotic system. Our method is a domain-independent approach based on an ontology as a knowledge base. We show how this generation can be performed on an ontology which is not dedicated to this task. We then validate our method through simulated situations, compare it with state of the art approach and on a real robotic system.

I. Introduction

Communication, be it verbal or not, is a key aspect for the success of a Human-Robot collaborative task. This becomes especially important in complex environments with a wide variety of entities: objects, places, people. Referring to a specific entity in such an environment can become challenging when one must account for the context of the task, the variety of facts that can be extracted from the situation depending on available perception modalities and the common ground between the robot and its human partners. The need to communicate on particular objects happens in many everyday tasks and it is important to endow the robot with the ability to estimate if and how referring to an object is feasible as well to assess the intelligibility of a referring expression (RE). This need is also important since deictic pointing is not always available in cooperative tasks when hands might be occupied to do something else.