Developmental Learning of Value Functions in a Motivational System for Cognitive Robotics

Motivation is quite an important topic when addressing continual open-ended learning processes in autonomous robots. The three main issues that need to be considered are, firstly, how does a designer define what the robot strives for in a manner that is independent from any particular domain it may find itself in. Secondly, once that robot is in a domain, how does it go about finding and relating goals in that particular domain on its own. Finally, the third issue is, once a goal is found, how does a robot establish a representation, usually in the form of a Value Function, that will allow it to exploit that goal. This paper deals with the third issue in the framework of the motivational engine we have designed for cognitive architectures. It addresses the problem of efficiently and appropriately learning complex Value Functions starting from intrinsically motivated traces of valuated robot actions that are often ambiguous and multivalued. To this end, a developmental learning mechanism is proposed that relies on the concurrent application of a real time ANN learning procedure over the traces of the valuated robot actions, and a simpler sensor correlation-based approach to allow for the production of better configured data traces for the learning process. The mechanism is analyzed and discussed over an experiment considering a real Baxter robot.