Applications of Reinforcement Learning in Three-phase Grid-connected Inverter

The grid-connected inverter is a key energy conversion device for grid-connected new energy and is widely used in distributed power generation system. However, the traditional control strategy has many limitations in the aspects of stability, system voltage and frequency adjustment, a large amount of renewable energy connected to the grid may cause power and frequency oscillation that threaten the stable operation of the grid. To solve this problem, an adaptive and optimal control method based on reinforcement learning and adaptive dynamic programming (ADP) is implemented for VSG three-phase grid-connected inverter. The method establishes a mathematical model of a VSG-based grid-connected inverter, and transformed into a standard linear quadratic regulation (LQR) optimization problem. On the basis of VSG power-frequency control, the dynamic compensation term given by the ADP algorithm is introduced into the active power loop. During the grid connection, the VSG output is optimally adjusted through the proposed adaptive optimal control strategy to reduce the system frequency fluctuation. In the case system dynamics are not known in the complex environment of grid-connected inverters, deep deterministic strategy gradient (DDPG) is used to replace the ADP algorithm in this paper. Finally, the effectiveness of the method is verified in the Simulink platform.