Reinforcement Learning-based Deep Queue Network Model for Tunnel Health Probing Protocol Management in SD-WAN

Software defined wide area network (SD-WAN) has dramatically eased granular level control over network resources on WAN edge devices. Current SD-WAN solutions employ a suitable tunnel health probing protocol (THPP) such as bidirectional forwarding detection (BFD) to periodically monitor the tunnel health. Conventionally, the periodicity of BFD probes is decided empirically and prescribed by SD-WAN vendors. However, owing to dynamic changes in the network conditions and difference in network environments in terms of the amount of resources invested, a universal and single probe frequency will not be optimal. Standard fixed value-based approaches are detrimental on network resources during congestion and yield sub-optimal network throughput. To overcome this shortcoming, this paper proposes a novel reinforcement learning (RL)-based deep queue network (DQN) model for tunnel health probe management that aims at achieving optimal network throughput for all network conditions. Simulation results validate the superiority of the proposed DQN model in adaptive management of the BFD probe frequency for dynamic network conditions.