Low-latency remote-offloading system for accelerator offloading

Specific workloads are increasingly offloaded to accelerators such as graphic processing unit (GPU) and field-programmable gate array (FPGA) for real-time processing and computing efficiency. Accelerators are expensive components and consume much power, so it is desirable to increase the efficiency of accelerator utilization by sharing accelerators among multiple servers over a network. However, task offloading over a network has the problem of latency due to network processing overhead in remote offloading. This paper proposes a low-latency system for accelerator offloading over a network. To reduce overheads of remote offloading, we propose a system composed of (1) fast recombination processing of chunked data with a simple protocol to reduce number of memory copy, (2) polling-based receiving data checking to reduce overhead due to interrupts in interaction with a network interface card, and (3) a run-to-completion model in network processing and accelerator offloading to reduce overhead with context switching. We show that the system can improve performance by 53.99% compared with a simple implementation and confirmed the performance with a virtual radio access network use-case as a low-latency application.