Improved Q-Learning Control for Optimal Tracking of Underwater Vehicle Manipulator System

In this work, an improved Q learning control algorithm base on an extended state observer (ESO) is designed for an underwater vehicle manipulator system (UVMS). The UVMS is modeled as a nonlinear system with model uncertainties and external disturbances. In this paper, extended state observer (ESO) is constructed to evaluate the model uncertainties and external disturbances. In addition, an augmented system is constructed based on UVMS and reference signal. Furthermore, an improved Q learning control method is proposed to solve online the augmented algebraic Riccati equation (ARE) in the absence of the knowledge of the augmented system parameters. Finally, extensive numerical simulation results show that the effectiveness of the proposed optimal tracking control method for UVMS.