Reinforcement Learning-Based Market Game Model Considering Virtual Power Plants

Virtual power plant (VPP) gathers various types of resources and is a new participant in the power market. The research on the market equilibrium state based on VPPs is helpful to analyze the problems existing in the construction of open power market. Based on the analysis of game behavior in power market, this paper constructs the decision-making models of VPPs and traditional power plants, and discusses the applicability of reinforcement learning in the bidding strategy of power producers. Based on this, a bi-level game model with VPPs is proposed. The model is solved iteratively based on the multi-agent reinforcement learning (MARL) method combining win or learn fast and policy hill-climbing algorithm, and the state space is compressed to improve the solution speed. Finally, through the example analysis, the results show that the proposed model and algorithm can effectively solve the optimal bidding strategy of VPPs and obtain the market equilibrium results. In the power market with constant total demand, VPPs have better resource integration and profitability than traditional power plants, and have higher adaptability to different power trading mechanisms.