Reinforcement Learning Based Controller for Grid-Connected PUC PV Inverter

Packed-U-Cell (PUC) is a single DC source multi-level inverter that can be used in many applications such as grid-connected photovoltaic (PV) systems. In this application, the total harmonic distortion (THD) of the generated current signal should be minimized while operating at the unity power factor (maximum active power transfer). These objectives can be achieved by regulating the auxiliary capacitor voltage around its reference value while tracking the reference current signal that varies with the PV maximum power point (MPP). Thus, this paper proposes a reinforcement learning (RL) based controller that satisfies the aforementioned control objectives using the actor-critic RL architecture and the proximal policy optimization (PPO) learning algorithm. The designed RL-based controller is applied on a single-phase 5-level PUC inverter. The proposed design is validated through simulations where the obtained control policy resulted in a maximum absolute voltage error of 1.9 V and a THD value of 2% and 4.5% for reference current peak values of 8.8A and 4.2A, respectively. Furthermore, the proposed RL-based controller shows high robustness to parameter variations (different capacitor and inductor sizes).