Deep Reinforcement Learning with Long-Time Memory Capability for Robot Mapless Navigation

Achieving autonomous navigation of indoor robots in a mapless environment is a long-standing research problem. Deep Reinforcement Learning (DRL) is widely used for robot navigation by virtue of learning through interaction with the environment. However, the large number of trials for training need lengthy computation times. To address this issue, we propose an innovative DRL model with long-time memory capability for mobile robot’s mapless navigation, which can achieve end-to-end navigation based only on laser-ranging data and target location. The long-time memory capability is realized by introducing a memory module based on the special structure of the Long Short-Term Memory (LSTM). The memory module ensures that the model derives information from previous navigation experiences and thus optimizes the model’s decision-making. In addition, to enable the model to explore the environment more effectively, we design a novel dual-noise mechanism consisting of Gaussian noise and Ornstein-Uhlenbeck noise. Extensive experiments are conducted on the Gazebo simulation platform and validate that the proposed approach can generate a smoother navigation path and exceed the state-of-the-art performance with less computation cost.