Trillium: The code is the IR

GPUs are the platform of choice for many general purpose workloads such as machine learning. This is driving demand for better GPGPU support in virtualized environments like the cloud. Despite significant research attention, GPGPU virtualization remains largely an open problem due to the challenge of balancing performance against key virtualization properties: compatibility, isolation, and interposition. Consequently, two different approaches to GPGPU virtualization have been adopted by the industry: Cloud service providers, such as AWS, support GPU-capable VMs using PCIe-passthrough techniques that bypass virtualization entirely, sacrificing its benefits; Virtualization vendors, such as BitFusion and Dell XaaS, support GPGPU virtualization using user-space API-remoting, which retains some of the benefits of virtualization, but elides hypervisor interposition, thereby giving up key virtualization properties.We hypothesize that while API-remoting may be the only viable software virtualization technique (as it interposes the only practical interface), API-remoting should not be implemented purely in user-space. We revisit VMware's SVGA in the context of GPGPU computing and find that hypervisor-mediated API-remoting is efficient: Decoupling device virtualization from GPU ISA virtualization is key to preserving the raw speedup from GPGPU acceleration, while also preserving the benefits of hypervisor-mediation: migration, isolation, fairness, etc.