Real-Time Operation of Energy Storage Assisting Utilization of Offshore Wind Power: Improving the Lyapunov Policy via DDPG

The uncertainty of wind power output and real-time electricity price poses challenges for the online operation of wind-storage integrated systems (WSIS). This paper proposes an advanced online dispatch algorithm for WSIS that combines Lyapunov Optimization (LO) and the Deep Deterministic Policy Gradient algorithm (DDPG). LO policy is regarded as the base policy, while DDPG provides an improved policy that is trained with LO operation data. A policy switching criterion is proposed to determine when to replace the LO policy with the DDPG policy for real-time operation. Case studies verify the effectiveness of the proposed method.