Robust H∞ Control of Unknown Discrete-Time Linear Systems with Time-Varying Uncertainties

The discrepancies between the mathematical model of control systems and the real dynamical model seriously affect the performance of the controller. This paper considers the
$H$

∞
control problem for unknown discrete-time linear systems with time-varying uncertainties. A robust controller is designed using on-policy and off-policy reinforcement learning (RL) methods. Interestingly, once the original linear system with uncertainties is transformed into an auxiliary system with a scale factor, the generalized
$H$

∞
control problem is turned into a scaled one. As a result, a new algebraic Riccati equation (ARE) is derived. When the dynamics of the system state equation are unknown, the on-policy RL is first introduced to solve the ARE, and then off-policy RL is used to learn a data-driven solution to the ARE. It is demonstrated that on-policy and off-policy RL methods have an equivalent convergence property.