Simultaneously learning at different levels of abstraction

Robotic applications in human environments are usually implemented using a cognitive architecture that integrates techniques of different levels of abstraction, ranging from artificial intelligence techniques for making decisions at a symbolic level to robotic techniques for grounding symbolic actions. In this work we address the problem of simultaneous learning at different levels of abstractions in such an architecture. This problem is important since human environments are highly variable, and many unexpected situations may arise during the execution of a task. The usual approach under this circumstance is to train each level individually to learn how to deal with the new situations. However, this approach is limited since it implies long task interruptions every time a new situation needs to be learned. We propose an architecture where learning takes place simultaneously at all the levels of abstraction. To achieve this, we devise a method that permits higher levels to guide the learning at the levels below for the correct execution of the task. The architecture is instantiated with a logic-based planner and an online planning operator learner, at the highest level, and with online reinforcement learning units that learn action policies for the grounding of the symbolic actions, at the lowest one. A human teacher is involved in the decision-making loop to facilitate learning. The framework is tested in a physically realistic simulation of the Sokoban game.

I. Introduction

Until not so long ago, robots were only confined to controlled industrial environments due to their limited capabilities to adapt to variable conditions. However, in recent years, advances in artificial intelligence [23], acting [11], and sensing [28] techniques are opening the door to robotic applications in less restricted environments, such as human environments [1]–[3]. Human-like tasks can be easily described in a symbolic way, using a notation fully compatible with AI techniques for planning and reasoning [7], [14]. These techniques are employed to make decisions using a declarative description similar to human language [2]. For instance, an action for picking and placing a cup from the cupboard to the table could be simply specified as pick_place (cup, cupboard, table). For the actual execution of the task, the symbolic description of actions should be properly grounded. This is carried out by a hierarchical decomposition of the task where, at each level of abstraction, a decision-maker determines the action (sub-task) to be performed by the lower level. For example, the pick and place action may be decomposed into several subtasks like reaching the cup, grasping it, lifting it, moving it to the table, and so on. Each of these sub-tasks may, in turn, be decomposed into other sub-tasks, like path planning to reach the cup. This goes down until the controller level, where the motor commands are sent to the robot. This hierarchical decomposition is carried out in a cognitive architecture that integrates different levels of abstraction, where each level defines the task to be carried out by the level below. The hierarchical task decomposition requires a proper interface to map the decision made at one level into a problem definition of the task for the level below. On the other hand, every decision is made using a model of the consequences of actions that allows the decision-maker to predict which action would be the best for the task at hand. This actually defines one of the main challenges in the implementation of the cognitive architecture. Even for apparently simple applications, coding all the possible consequences of actions in all the possible situations, and at all the levels of abstractions, might be a big burden and many important aspects can be easily overlooked. Furthermore, in variable environments, even when complete models are provided, unexpected variable conditions found during task executions would require these models to be updated. Therefore, the implementation of such cognitive architecture is only feasible if, at each level, a learning mechanism that updates the action model is provided. Even more, to avoid long task interruptions every time a model at any level have to be updated, learning should take place simultaneously at all levels and the action models should be updated in a reasonable time.