Visual Kinematics Representation via Self-Supervised Object Tracking for Deep Reinforcement Learning

Feature representation is critical for deep reinforcement learning (RL) in terms of sample efficiency and asymptotic performance. Previously developed feature representation methods in deep RL, such as self-supervised learning, only leverage the properties of raw-pixel signals without explicitly mining semantic information of tasks. In this paper, we propose a novel Visual Kinematics Representation based RL (VKR-RL) approach that effectively transforms original raw-pixel signals into dynamic objects kinematics as feature representations for RL training. VKR-RL involves a self-supervised object tracker for discovering and tracking dynamic objects, and obtained the visual kinematics representations by parsing the tracking results. Simulations conducted on two Atari tasks (i.e., Tennis and Pong) show that VKR-RL can achieve better sample-efficiency, asymptotic performance, and generalization capability than the deep RL approaches with previously developed feature representation learning methods.