Deep Reinforcement Learning-Based 3D Exploration with a Wall Climbing Robot

It is crucial to impart autonomy to robots for efficient exploration in surveillance and military applications. Most research on exploration deal largely with 2D environments using ground-based robots or 3D environments using drones. For applications requiring stealth, wall-climbing robots have an edge over drones however research is scant in 3D exploration algorithms using such robots. This paper presents a constrained 3D mapping problem and proposes a reinforcement learning-based exploration algorithm for the same using a lizard-inspired robot. The developed approach is based on a simulation-based framework in CoppeliaSim for simulating robot motion and Tensorflow for action selection algorithm using Deep Q Network (DQN). We report a number of simulation results indicating an improvement in median coverage by 20.3% using the developed DQN-based action selection over that of one generated by random action selection. Moreover, we observe that the developed DQN-based action selection approach led to more than 80% coverage within 150 steps, whereas the random action selection approach barely exceeded 60% coverage. We envisage that the developed approach can help imparting autonomy to the wall climbing stealthy robots in hostage scenarios.

I. Introduction

Wall climbing robots have several applications in surveillance [1] and inspection [2]. Being mobile, these robots can cover more area than fixed CCTV cameras. Also instead of several fixed cameras, relatively few wall climbing robots can cover large areas. In cases of emergencies (terrorist attacks, natural disasters), these robots can be deployed in places that lack surveillance cameras.