Autonomous robot navigation system with learning based on deep Q-network and topological maps

This paper proposes a autonomous mobile robot navigation system integrating local navigation based on deep reinforcement learning and global navigation based on topological maps. Especially, the proposed system aims to achieve a robot navigation that is adaptive with human traffic in environments crowded with many people. In order to obtain abilities for adapting dynamic obstacles in such environments, a simulator imitating real worlds is required for learning. Then a simulator suitable for learning of robot navigation in crowded environments is developed at first. This simulator is specialized in simulation of two-dimensional shape, and it could achieve learning in shorter time than existing simulators. Next, a learning system based on DDQN was designed for local navigation with avoiding pedestrians. Some simulations and experiments were performed to present effectiveness of the proposed navigation system. Simulation results show that the system learned abilities to go to local destinations with avoiding dynamic obstacles Also, the system was applied to an actual robot system and navigation in a real world was demonstrated.