An Adaptive Dynamic Programming Algorithm Based on ITF-OELM for Discrete-Time Systems

Adaptive dynamic programming (ADP) is a kind of intelligent control method, and it is a non-model-based method that can directly approximate the optimal control policy via online learning. The gradient algorithm is usually used to update weights of action networks and critic networks, however it is clear that gradient descent-based learning methods are generally very slow due to improper learning steps or may easily converge to local minimum. In this paper, in order to overcome those disadvantages of gradient descent-based learning methods, a novel ADP algorithm based on initial-training-free online extreme learning machine (ITF-OELM), in which the critic network link weights of hidden nodes to output nodes can be obtained by least squares instead of gradient algorithm, is introduced. Finally, the ADP algorithm based on ITF-OELM is tested on a discrete time torsional pendulum system, and simulation results indicate that this algorithm makes the system converge in a shorter time compared with the ADP based on gradient algorithm.

1 Introduction

Model based control techniques have been developed in order to cope with control problems on the assumption that models of the controlled systems are known, the production equipment is becoming increasingly complicated, modeling a system is not easy, and sometimes it is impossible. It is a very meaningful to study non-model-based control methods for unknown discrete time control systems. Adaptive dynamic programming (ADP) [1]–[5] is a kind of intelligent control method, and it can directly approximate the optimal control policy via online learning. Heuristic dynamic programming (HDP), dual heuristic programming(DHP), action dependent heuristic dynamic programming(ADHDP), and action dependent dual heuristic programming (ADDHP) are four basic adaptive dynamic programming structures [6]. HDP is a typical ADP, it was proposed in the 1970s, and the idea was firmed up in the early 1990s under the names of adaptive critic designs.