Automated Traffic Engineering in SDWAN: Beyond Reinforcement Learning

Traffic engineering (TE) is a critical and difficult problem that involves assigning traffic with various requirements to paths with different constraints. Recently, machine learning algorithms, especially deep neural networks (DNN), are applied to TE, yet they all assume that the network is a black box, limiting them to only model-free reinforcement learning (RL) algorithms. In this paper, we introduce differentiable programming to TE, and show that the network environment can be sufficiently modeled for TE optimization. Specifically, we design a fully-differentiable network environment, ∂NE, that can be directly integrated into any DNN models. With ∂NE, we can differentiate with respect to control parameters, and directly evaluate gradients between actions and states to facilitate gradient descent based training of DNN models. We show with a proof-of-concept prototype that ∂NE accelerates DNN training for TE by 228× and achieves higher scalability compared to existing network simulators. Most importantly, dNE opens up the possibility to apply arbitrary deep learning models to TE beyond RL.