Multi-Tier Workload Consolidations in the Cloud: Profiling, Modeling and Optimization

Reducing tail latency becomes increasingly important to improve the user-perceived service experience. User-facing latency-sensitive cloud applications typically contain multiple interactive tiers (e.g., Web, App, Database) running in different virtual machines (VMs) with complex interaction patterns. However, such interactions between VMs in different tiers are often neglected in previous VM consolidation methods, resulting in poor application performance. In this article, we study the consolidation of multi-tier interactive workloads from a new perspective of user-perceived tail latency. We propose a novel profiling-based consolidation methodology to satisfy tail latency requirements while reducing the number of used physical machines. To achieve such a goal, we first perform large-scale profiling experiments under various consolidation settings in a KVM virtualized private cluster to establish the empirical performance values. We consider two key factors that affect the tail latency of multi-tier workloads: interference with co-located VMs and interaction between tiers. We model the consolidation of multi-tier workloads as an optimization problem with different objectives and constraints, and derive the consolidation schedule. We implement and evaluate the proposed models, as well as comparing with other methods (i.e., without profiling or without considering interaction influence). Extensive experimental results show that the proposed method is able to reduce up to 5X tail latency, compared with the method without profiling and up to 1.3X tail latency, compared with the method without considering the interaction influence between different tiers.

1 Introduction

Cloud computing provides a novel computing and storage model for IT services that offers opportunities not only to on-demand resource provision but also to better manageability. Virtual machine (VM) is the basic computing unit in a typical IaaS cloud. Server consolidation enabled by virtualization is a key technique to improve the resource utilization of physical machines (PMs) by consolidating multiple workloads (i.e., applications) running in virtual machines onto the same PM. By doing so, the server utilization is increased and the power consumption is reduced. Due to these and other benefits, today many enterprises use cloud computing to deploy their online services.