A Q-learning-based Multi-timescale Resilience Enhancement Approach for Power Grids with High Renewables

Power grid resilience is critically undermined by extreme weather events, especially when coupled with uncertain renewable energy generation, which can lead to a severe power outage. When facing such outages, it is essential for system operators to restore damaged lines in a sequence that maximizes the overall power supply. Identifying the optimal restoration sequence can be cast as a multi-stage problem where at each stage a single line is first restored in an hourly basis and then numerous power flow problems within the timescale of minutes are optimized under the resulting network topology. Evaluating the economic effect of restoring a specific line entails the optimization of power flow problems with time-varying renewables, lacking a closed-form expression. To address this, we employ the Q-learning algorithm, utilizing a well-designed Q-function to approximate the effect of restoring each line and embedding such a function into a training process. Simulations on the IEEE 33-Bus and IEEE 39-Bus Systems confirm the efficacy of the proposed approach, showcasing significant enhancements in resilience and recovery capabilities.