Costmap-based Local Motion Planning using Deep Reinforcement Learning

Local motion planning is an essential component of autonomous robot navigation systems as it involves generating collision-free trajectories for the robot in real-time, given its current position, the map of the environment and a goal. Considering an a priori goal path, computed by a global planner or as the output of a mission planning approach, this paper proposes a Two-Stream Deep Reinforcement Learning strategy for local motion planning that takes as inputs a local costmap representing the robot’s surrounding obstacles and a local costmap representing the nearest goal path. The proposed approach uses a Double Dueling Deep Q-Network and a new reward model to avoid obstacles while trying to maintain the lateral error between the robot and the goal path close to zero. Our approach enables the robot to navigate through complex environments, including cluttered spaces and narrow passages, while avoiding collisions with obstacles. Evaluation of the proposed approach was carried out in an in-house simulation environment, in five scenarios. Double and Double Dueling architectures were evaluated; the presented results show that the proposed strategy can correctly follow the desired goal path and, when needed, avoid obstacles ahead and recover back to following the goal path.