Obstacle-free Trajectory Planning of an Uncertain Space Manipulator: Learning from a Fixed-Based Manipulator

In a typical space debris mitigation mission, a space manipulator must plan maneuvers free of self-collision and collision with the noncooperative target satellite to perform a safe capture. We develop an effective model-free planner based on deep reinforcement learning for free-floating manipulators that only relies on an uncertain target position feedback. At its core, the learning agent employs the Deep Deterministic Policy Gradient (DDPG) algorithm capable of working with continuous states and actions. To improve the learning performance, we propose a five-step sequential learning that uses priority episode sampling to effectively transfer knowledge from a fixed-based manipulator trained to follow a moving target to an uncertain space manipulator capturing a target point on a satellite. Further, to avoid moving obstacles, we introduce the notion of multi-critic in the DDPG setting, such that one critic optimizes the task of chasing in an uncertain environment and another one focuses on obstacle avoidance. To show the efficacy of the developed trajectory planner, we compare its running average success rate and reward value with a baseline DDPG.