Regret Analysis for Learning in a Multi-Agent Linear-Quadratic Control Problem

We consider a multi-agent Linear-Quadratic (LQ) reinforcement learning problem consisting of three systems, an unknown system and two known systems. In this problem, there are three agents - the actions of agent 1 can affect the unknown system as well as the two known systems while the actions of agents 2 and 3 can only affect their respective colocated known systems. Further, the unknown system's state can affect the known systems' state evolution. In this paper, we are interested in minimizing the infinite-horizon average cost. We propose a Thompson Sampling (TS)-based multiagent learning algorithm where each agent learns the unknown system's dynamics independently. Our result indicates that the expected regret of our algorithm is upper bounded by Õ(√T) under certain assumptions, where Õ(·) hides constants and logarithmic factors. Numerical simulations are provided to illustrate the performance of our proposed algorithm.