SDSSE: A Self-Driven RPC Load Balancing Method in Datacenter

Remote Procedure Calls (RPC) load balancing is crucial to maximize server resource utilization and reduce tail latency in data center. Existing load balancing methods deploy a frontend server acting as a load balancer, responsible for requesting backend server workload and routing RPC requests to backend server. However, existing methods suffer from two issues. Firstly, they introduce additional latency to RPC by requiring the frontend server to request the resource utilization of backend servers. Secondly, as the number of RPC requests grows, the frontend server becomes a bottleneck in the system, limiting the scalability of the load balancing approach. Programmable switches have become an effective way to serve high traffic rates involving many concurrent flows. In this paper, we propose a novel RPC load balancing approach that enables the programmable switch to take over the RPC request load balancing function. By maintaining a local estimation of the distribution of the backend server workload, the switch can directly guide the selection of a suitable backend server. We formulate optimizing server selection policy as a Bandits with Knapsack (BwK) problem. Our objective is to strike a balance between maximizing the cumulative utilization of server resources and minimizing cumulative latency. Extensive experiments in diverse scenarios demonstrate that the proposed method can effectively improve server selection policy and boost RPC load balancing performance.