Evolutionary Deep Reinforcement Learning for Volt-VAR Control in Distribution Network

As an important form of renewable energy integrated to the power system, distribution network is being challenged by voltage violation and network loss increase. Currently, model-based Vol-Var control (VVC) methods are widely used to reduce voltage violation and network loss. However, model-based methods need accurate parameters of distribution network. In practice, accurate model is difficult to obtain. In this paper, we propose a model-free evolutionary deep reinforcement learning (E-DRL) algorithm to solve the VVC problem. Based on E-DRL, the agent evolves autonomously by continuously interacting with the environment learning control strategy. Inverter-based PVs and SVGs are used to provide fast and continuous control. VVC problem is solved by soft actor-critic algorithm, which uses the maximum entropy technique to balance the exploration and exploitation. Numerical simulations on IEEE 13-bus system demonstrate that the proposed method has satisfied performance.

I. Introduction

In order to cope with the serious environmental problems, China has proposed to achieve the goal of emission peak by 2030 and carbon neutrality by 2060. Building a new power system based on renewable energy is an important way to achieve the goal of emission peak and carbon neutrality. Distribution network is an essential form of renewable energy integrated to the power system. However, the generation of renewable energy is uncertain. The integration of a large amount of distributed renewable energy source (RES) brings great challenges to the operation of distribution network. It causes a series of problems such as harmonics, voltage violation and network loss increase, which would reduce the safety and economy of distribution network [1]. Among them, the voltage problems are particularly prominent. In distribution network, the resistance to reactance ratio of line conductors is high to make the fluctuation of RES have a significant impact on node voltage. Therefore, the Volt-VAR control of distribution network is essential.