Infusing ML into VM Provisioning in Cloud

In cloud computing, provisioning virtual machines (VMs) fast and reliably is a fundamental yet challenging problem, particularly in cloud environments of changing workloads and shifting demand patterns. A trivial solution of instantiating a VM from scratch per customer request may not satisfy business SLA's and would degrade customer experience. Hence, provisioning VMs in advance motivated with machine learning (ML) infused into the cloud computing system to predict upcoming VM request demands is an advisable solution. In this paper, we first describe a number of system integration challenges including 1) how to achieve low latency provisioning to quickly adjust to customer demand pattern shifts, 2) how to efficiently scale to serve a large number of VM configurations supported in the cloud environment, and 3) how to reliably consume recommended prediction results for VM provision despite of anticipated operation failures and timeout. We then present the high level solution design with discussions of our developed system to address the aforementioned challenges. Our system has been implemented and deployed successfully in Microsoft Azure exhibiting significant improvements for VM provisioning experience with regards to latency and reliability requirements.