Optimization of CPS Control for Double Deep Q Learning Based on Function Regularization

Markov decision process (MDP) is a decision model based on probability theory and control theory, which is widely used in the field of automation control. Automatic generation control is a common MDP issue in the linked power grid. Q-learning is a reinforcement learning algorithm based on MDP, which can achieve dynamic optimization decision-making and online learning. The NERC's control performance standard(CPS) value may be employed as a reward signal to optimize the strategy since the automatic generation control(AGC) system is an uncertain stochastic system. Moreover, replacing Bellman error with function regularization may increase the AGC system's flexibility and resilience while successfully avoiding overfitting issues. Through simulation experiments, this paper proves that the double deep Q-learning algorithm with function regularization can significantly improve the CPS pass rate, and has practical application value.