A Brain-Inspired Incremental Multitask Reinforcement Learning Approach

Recently, there have been growing interests in multitask reinforcement learning (MTRL), which is viewed as a promising framework for training agents to execute multiple tasks simultaneously. However, limitations in scalability and convergence remain key obstacles for scaling these MTRL algorithms to dynamic and complex tasks. To address these, we propose a method called brain-inspired incremental multitask reinforcement learning (BIMTRL) that aims to improve parallelism and scalability of multiple tasks. Inspired by learning processes in human brain, we integrate conscious and subconscious modes into the agents’ exploration of environments. Our two-step strategy of
policy loosening
and
importance tradeoff
enables an effective switch between these modes. Additionally, in order to overcome the convergence dilemma, we adopt the
V-trace
method as a stable and robust off-policy correction technique for our actor-critic agents. Experimental evaluations on various tasks in OpenAI Gym, Atari, and PyBullet have demonstrated that BIMTRL achieves a 61.4% greater average return and 57.6% higher speed than specific multitask baselines. Furthermore, its distributed and incremental architecture endows the BIMTRL approach with a desired scalability in both discrete and continuous environments, ultimately leading to larger rewards, higher speed, and better convergence.