Malta: Multi-Agent Reinforcement Learning for Differentiated Services in Fat Tree Networks

Fat tree topologies have been gaining traction in datacenter networking due to the benefits of scalability, efficiency and fault resilience. Fat tree networks typically employ Equal Cost Multi-Path (ECMP) routing techniques for traffic load balancing. However, ECMP techniques are sub-optimal at distinguishing and providing differentiated services to various flows, which is a necessary requirement for 5G networks. In this paper, we propose Malta, a Multi-Agent Reinforcement Learning technique to provide differentiated service guarantees in fat tree networks. Multi-agent reinforcement learning techniques offer scale, flexibility in reward structure and can be used to learn optimal behaviour with respect to differing traffic patterns. We demonstrate the utility of such agents over a real use case involving multiple flows with heterogeneous actions at the leaf, spine and super-spine level. The efficacy of the approach is shown in resolving congestions at the spine and super-spine level, that are unable to be resolved by ECMP. In addition, Malta is shown to provide superior differentiated service guarantees with 46% latency improvement and 34% throughput improvement over vanilla ECMP.