Software-Defined Routing Strategy Based on Reinforcement Learning in Smart Power Grid

To enable a set of advanced functionalities, the smart power grid relies on pervasive sensors and actuators for continuous monitoring and control. These field devices handle both regular and event-driven (emergency) data packets, which require different levels of quality-of-service (QoS) support (latency requirements). Unfortunately, the existing routing strategies in smart power grids do not guarantee differentiated QoS support that adapts to the network dynamics. To address this limitation, we propose a QoS-aware routing strategy in smart grids based on a software-defined network (SDN). The proposed framework establishes separate queues for event-driven and fixed-scheduling packets. Then, the routing strategy is designed based on reinforcement learning to enable adaptive QoS support. In this regard, updating the routing tables at the OpenFlow switches for each incoming packet is not practical and incurs additional delays that would violate the target QoS. Instead, the proposed routing strategy relies on two Q-learning agents that fix an optimal route for each source-destination pair and update the queue service rate for each switch along the route depending on the network condition (packet arrival rate). Our simulation results demonstrate that the proposed adaptive strategy offers effective QoS support compared with an SDN benchmark routing strategy based on Bellman-Ford. For instance, the average percentage of packets that violate their latency requirement is only 3% compared with at least 30% for the benchmark routing strategy.