Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems

This paper studies the robust optimal control design for a class of uncertain nonlinear systems from a perspective of robust adaptive dynamic programming (RADP). The objective is to fill up a gap in the past literature of adaptive dynamic programming (ADP) where dynamic uncertainties or unmodeled dynamics are not addressed. A key strategy is to integrate tools from modern nonlinear control theory, such as the robust redesign and the backstepping techniques as well as the nonlinear small-gain theorem, with the theory of ADP. The proposed RADP methodology can be viewed as an extension of ADP to uncertain nonlinear systems. Practical learning algorithms are developed in this paper, and have been applied to the controller design problems for a jet engine and a one-machine power system.

I. Introduction

Reinforcement learning (RL) [38] is an important branch in machine learning theory. It is concerned with how an agent should modify its actions based on the reward from its reactive unknown environment so as to achieve a long-term goal. Originally observed from biological learning behavior, RL has been brought to the computer science and control science literature as a way to study artificial intelligence in the 1960s [28], [29], [45]. Since then, numerous contributions to RL have been made; see [4], [39], and [47].