$H_{\infty}$ Control for Interconnected Systems With Unknown System Dynamics: A Two-Stage Reinforcement Learning Method

This paper investigates the optimal control problem for interconnected systems with unknown system dynamics through a two-stage reinforcement learning method. First, to address the impact of interconnection term, the optimal control problem for interconnected systems is transformed into obtaining the solution of game algebraic Riccati equations within the framework of
$H_{\infty}$
control method. Furthermore, existing optimal control approaches for interconnected systems necessitate precise knowledge of system dynamics, which is difficult to obtain accurately or involves high costs. Thus, we introduce a two-stage reinforcement learning method. The admissible control policies are obtained using the homotopy-based iteration method in the first stage. Then, the optimal control policies are obtained through the policy iteration method in the second stage. The two-stage method not only eliminates the requirement for system dynamics and initial admissible control policies but also ensures convergence speed and accuracy, significantly enhancing its practicality. Finally, a two-machine power system example is provided to validate the feasibility of the two-stage method.
Note to Practitioners
—Interconnected systems, a class of systems composed of multiple local subsystems, find wide applications in various fields such as power systems, transportation networks, and spatially interconnected systems. Particularly, the optimal control problem of interconnected systems has gradually become a focal point of current research. However, the current research on the optimal control problem of interconnected systems is still constrained by the system dynamics and the initial stability of the system. To relax these limitations, this paper introduces a two-stage method. A homotopy-based iteration approach is employed to obtain control policy and interconnection policy that make the system closed-loop stable, thus achieving the optimal solution. Furthermore, the data-driven approach overcomes the limitations imposed by system dynamics. The feasibility of the two-stage method is illustrated by a two-machine power system model.