Initial Excitation-Based Iterative Algorithm for Approximate Optimal Control of Completely Unknown LTI Systems

This paper proposes an approximate/adaptive optimal control (AOC) design for completely unknown continuous-time linear time invariant systems, without requiring the restrictive persistence of excitation (PE) condition for parameter convergence. The proposed AOC algorithm utilizes two layers of filtering—the first layer filters strategically eliminate the need for state derivative information, while the second layer filters provide suitable algebraic relations for iteratively obtaining the optimal policy under a milder online-verifiable initial excitation assumption. Unlike previous AOC algorithms, the proposed method does not require finite window integrals, intelligent data-storage, and the restrictive PE assumption. Further, the proposed method relaxes the sufficient condition required for obtaining successive stabilizing control policies. The intermediate policies are proved to be stabilizing and converging to the optimal policy. Simulation results validate the efficacy of the proposed adaptive/approximate linear quadratic regulator algorithm.