Balancing Load: An Adaptive Traffic Management Scheme for Microservices

Service Mesh has become one of the most popular microservices governance frameworks. In the service governance function of traffic management, how to design an efficient strategy for request distribution to minimize response time has attracted wide research interest. Generally, a traffic policy can be pre-set manually. However, such a static policy always performs terribly when the load changes dynamically. That is, Service Mesh cannot adaptively select the optimal policy when setting service version diversion and selecting instances on the service subset. To address this issue, in this paper, we designed an adaptive strategy based on split-flow to achieve global load balancing. Furthermore, we implement such a strategy by developing a plug-in named DTMA for Istio, a typical representative of Service Mesh. By obtaining the Service Mesh node’s topology and global instance load distribution in real-time, Istio updates the optimal split-flow weight and selects the suitable load balancing strategy. Extensive experimental results show that our algorithm can reduce the response time by nearly half and keep a load of nodes and instances stable compared with Istio’s native configuration. The optimization effect of minimizing the response time is achieved through real-time load balancing.