Deep Reinforcement Learning-based Multi-Scenario Multi-Scale Economic Scheduling for Integrated Energy Systems

The integrated energy system (IES) with multi-energy coupling and gradient utilization has become an important means to help realize the goal of “double carbon”. The optimal operation of IES is a nonlinear, nonconvex and complex problem. With the gradual increase of intermittent energy and load uncertainty, traditional scheduling methods are limited to fixed physical models and parameter settings, which make it difficult to quickly and accurately obtain optimal scheduling strategies. In this paper, we propose a distributed proximal policy optimization (DPPO) algorithm to train the IES optimal scheduling model offline and solve the optimization online, which transforms the physical scheduling problem into a reinforcement learning sequential decision-making problem, and adaptively learns the scheduling policy through continuous interaction between the agent and the integrated energy system environment, and the multi-threaded structure enhances the exploration ability of the reinforcement learning agent in the complex environment. Example simulations show that online applications do not need to rely on accurate modeling of the physical model of the IES to achieve fast scheduling decisions in multiple scenarios and scales.