MIMIC: SmartNIC-aided Flow Backpressure for CPU Overloading Protection in Multi-Tenant Clouds

In multi-tenant clouds, off-the-shelf x86 boxes are widely deployed as middleboxes. With the rapid growth of cloud traffic and the migration to NFV deployment in recent years, CPU overloading at middleboxes becomes more of an issue. From our data centers, we observed that the CPU overloading was caused by heavy hitters. To address this issue, we propose MIMIC, a cloud-scale flow backpressure system, implemented onto our existing SmartNIC with FPGA acceleration. MIMIC rate-limits the selected heavy hitters through a new per-flow backpressure protocol and a new heavy-hitter detection system, to protect the other tenants. The detection system is based on hierarchical memory design, leveraging on-chip SRAM and off-chip DRAM, which can handle highly concurrent cloud traffic without the losses of flow information. We extend the design by adding a pre-filtering procedure for rapid detection. To avoid CPU being flooded by FPGA through frequent heavy-hitter reporting, due to their performance disparity, the CPU queries the FPGA on demand. The backpressure protocol is non-invasive to protect tenant privacy and allows controllable rate-limiting through the novel use of ECN and meter tables. The SmartNIC acts as a man in the middle to facilitate heavy-hitter detection and per-flow backpressuring. In a production setting, we observe that MIMIC can react quickly and bring down CPU load to the normal level within 10ms without packet losses.

I. Introduction

Cloud services have been increasingly adopted by organisations and individuals [1]. This trend is accelerated by the roll-out of 5G mobile networks and the pandemic. Cloud vendors are motivated to provide a diverse range of services appealing to customers. As a major cloud vendor, we have observed the rapid growth of cloud traffic in recent years. We start experiencing more often CPU overloading at our NFV-based middleboxes on the path of east-west traffic (VM-VM) (Fig. 3). By looking into individual CPU overloading occurrences, we found that CPU overloading was caused by heavy-hitter flows (Top-1 flows in Fig. 2). In a multi-tenant cloud, due to infrastructure sharing, CPU overloading may lead to packet losses and long latency across all the tenants whose traffic goes through the CPU, even if the majority of the tenants do not contribute to the heavy hitters.