Deterministic Reinforcement Learning Consensus Control of Nonlinear Multi-Agent Systems via Autonomous Convergence Perception

This brief addresses an approximate optimal consensus control problem for multi-agent systems (MAS) using an autonomous perception empowered deterministic reinforcement learning scheme. An autonomous perception module is designed to online detect the convergence of NNs and tracking performance, and determine proper time moments that reinforcement learning (RL)-based method can be switched to deterministic RL (DRL) mode with constant NNs weight vectors, saving computing resources and reducing computational amount for devices equipped on MAS. Furthermore, once non-convergence of NNs or consensus errors is detected again, the designed autonomous perception module would switch DRL back to RL mode without chattering in the closed-loop system, though switch behaviour and non-smooth function are involved. Simulation results are finally presented to demonstrate the effectiveness.

I. Introduction

Consensus control of multiple agents in a group has been a hot issue in the recent decades due to its potentially practical applications in various physical plants, such as mobile sensor networks, cooperation control of multiple vehicles, and so on [1], [2], [3]. Some pioneering and significant results have been reported in existing literature. To name a few, in [4], necessary theoretical analysis of nearest neighbor rules-based coordination of autonomous agents is given. In [5], consensus algorithm under dynamically changing interaction topologies is achievable if a spanning tree exits in directed interaction graphs. Inspired by this success, in [6], a distributed consensus control algorithm is designed for a class of higher order agent dynamics and dynamically changing directed interaction topologies. In [7], an adaptive consensus algorithm is designed for a class of nonlinear MAS via neural networks (NNs) approximation of unknown agent functions. In [8], an optimized backstepping consensus control algorithm is designed using reinforcement learning (RL) to achieve the optimal performance for a class of nonlinear MAS with undirected graphs. In [9], a dual event-triggered constrained control is proposed for nonlinear zero-sum games by adaptive critic. In [10], a novel integrated multi-step heuristic dynamic programming algorithm is proposed for solving optimal control with stability guarantee.