Optimal Control for Unknown Nonlinear System With Semi-Markovian Jump Parameters via Adaptive Dynamic Programming

This article investigates the optimal control problem for the discrete-time (DT) nonlinear semi-Markovian jump systems (s-MJSs) that possess unknown dynamics. The study uses the semi-Markovian kernel approach to address the problem of mode-switching in these systems. This approach employs the transition probability and the sojourn-time distribution function to jointly determine the transitions between different modes. Then, with a neural network (NN) identifier, the demand for accurate information on the system dynamics is eliminated, and an optimal control method for the nonlinear s-MJSs is utilized to solve the Hamilton-Jacobi–Bellman equation (HJBE) built upon adaptive dynamic programming methodology. Additionally, a detailed analysis of the convergence of a value iteration-based algorithm, which solves the optimal control issue for the DT s-MJSs, is thoroughly discussed. Furthermore, an actor-critic NN is trained to attain an estimated solution to the relevant HJBE. Finally, to validate the designed approach, two simulations are performed to prove its effectiveness.