Fast stochastic motion planning with optimality guarantees using local policy reconfiguration

This work presents a framework for fast reconfiguration of local control policies for a stochastic system to satisfy a high-level task specification. The motion of the system is abstracted to a class of uncertain Markov models known as bounded-parameter Markov decision processes (BMDPs). During the abstraction, an efficient sampling-based method for stochastic optimal control is used to construct several policies within a discrete region of the state space in order for the system to transit between neighboring regions. A BMDP is then used to find an optimal strategy over the local policies by maximizing a continuous reward function; a new policy can be computed quickly if the reward function changes. The efficacy of the framework is demonstrated using a sequence of online tasks, showing that highly desirable policies can be obtained by reconfiguring existing local policies in just a few seconds.

I. Introduction

The objective for traditional robotic motion planning is to compute a trajectory that will move the robot between two valid poses while respecting all physical constraints [1] – [3]. In practice, however, robots suffer from unexpected events like noisy actuation of a wheeled base when navigating uneven terrain, imperfect observations due to unfavorable sensing conditions, or an incorrect map if objects have moved around a room. These kinds of disturbances force the robot to deviate from its current course into a state from which there may be no clear path to the goal. Replanning is one option to address these detours, but this can be computationally prohibitive if the deviations are frequent or constant. A more robust strategy to combat motion planning under uncertainty is to model the problem as a stochastic decision process where the solution is not a single trajectory, but rather a control policy over all possible states of the system to maximize an objective function [3], [4].