An Intelligent PID Controller for Ships Based on Soft Actor Critic Algorithm

Proportional integral differential (PID) controllers are widely used in industry due to their practicality and good control effects. However, the three controller gains need to be manually tuned based on experience. Sometimes, improper gain settings led to poor control performance, which limits the application of PID controllers. Moreover, the adjusted PID gains are fixed and cannot be modified in real-time. In order to overcome the aforementioned shortcomings, the Soft Actor Critic (SAC) algorithm is used to adaptively adjust gains based on the “trial and error” characteristics of deep reinforcement learning algorithm. After sufficient iterative training, a set of good PID gains are obtained, and an intelligent ship controller is designed. The final experimental results indicate that the proposed adaptive PID gains tuning method based on deep reinforcement learning not only achieves better intelligent ship heading and trajectory control performance, but also has good robust performance.