Performance Evaluation of VirtIO Device Drivers for Host-FPGA PCIe Communication

Unleashing the full potential of FPGAs as offload devices requires Host-FPGA connectivity that is reliable, robust, and uniform; and that implements (at least) a required set of features and protocols. PCIe is the most widely used host-FpgA interface for high-performance applications. However, existing frameworks for host-FpgA PCIe communication have several limitations, including lack of portability and poor upstream support. Native VirtIO drivers in the host operating system can address many of these limitations on the host side. A complete VirtIO-based solution, however, also requires new support on the device side. In previous work, a general framework was proposed that requires little additional programming effort per new device. Although VirtIO drivers could provide an attractive alternative to vendor-provided device drivers, their performance when interacting directly with physical devices has not been explored. Given that VirtIO drivers are designed targeting virtual devices, it is critical to investigate whether they perform at an acceptable level when handling physical devices. In this work, we compare the performance of VirtIO device drivers to vendor-provided device drivers in terms of communication latency and latency distribution and show VirtIO drivers provide similar or slightly improved performance with reduced variance. To facilitate our analysis, we also extend prior work in implementing support for VirtIO network devices on FPGAs. The overall significance of this work is that it demonstrates the feasibility of replacing the vast space of legacy device drivers with the VirtIO drivers already native to the host OS.