Imitation-Based Reinforcement Learning for Markov Jump Systems and Its Application

In this paper, the imitation reinforcement learning-based control problem is studied for discrete-time Markov jump systems with external disturbances. First, zero-sum game method is introduced to deal with external disturbances, where control input and external disturbances are regarded as two rival players in adversarial environments. Then, the imitation reinforcement learning problem is formulated, where learner Markov jump systems aim to learn the optimal behavior of expert Markov jump systems. Considering that the dynamics information of both learner systems and expert systems is accurately known, an offline parallel imitation learning algorithm is designed for learner systems to mimic expert behaviors, which contains three steps: 1) policy evaluation, 2) search for weight matrix, and 3) policy improvement. On this basis, by observing the optimal behavior of expert systems, an online imitation reinforcement learning algorithm is presented for learner systems with completely unknown system dynamics. Moreover, rigorous proofs of convergence and stability analysis are provided to guarantee the performance of proposed algorithms. Finally, the effectiveness of the proposed method is verified by a single-machine infinite-bus power systems.