μ Suite: A Benchmark Suite for Microservices

Modern On-Line Data Intensive (OLDI) applications have evolved from monolithic systems to instead comprise numerous, distributed microservices interacting via Remote Procedure Calls (RPCs). Microservices face single-digit millisecond RPC latency goals (implying sub-ms medians)-much tighter than their monolithic ancestors that must meet ≥ 100 ms latency targets. Sub-ms-scale OS/network overheads that were once insignificant for such monoliths can now come to dominate in the sub-msscale microservice regime. It is therefore vital to characterize the influence of OSand network-based effects on microservices. Unfortunately, widely-used academic data center benchmark suites are unsuitable to aid this characterization as they (1) use monolithic rather than microservice architectures, and (2) largely have request service times ≥ 100 ms. In this paper, we investigate how OS and network overheads impact microservice median and tail latency by developing a complete suite of microservices called μSuite that we use to facilitate our study. μSuite comprises four OLDI services composed of microservices: image similarity search, protocol routing for key-value stores, set algebra on posting lists for document search, and recommender systems. Our characterization reveals that the relationship between optimal OS/network parameters and service load is complex. Our primary finding is that non-optimal OS scheduler decisions can degrade microservice tail latency by up to ~87%.

I. Introduction

On-Line Data Intensive (OLDI) applications such as web search, advertising, and online retail form a major fraction of data center applications [105]. Meeting soft real-time deadlines in the form of Service Level Objectives (SLOs) determines end-user experience [19], [47], [57], [90] and is of paramount importance. Whereas OLDI applications once had largely monolithic software architectures [51], modern OLDI applications comprise numerous, distributed microservices [69], [87], [108], [117] such as HTTP connection termination, key-value serving [73], query rewriting [49], click tracking, access-control management, protocol routing [22], etc. Several companies such as Amazon [3], Netflix [26], Gilt [12], LinkedIn [20], and SoundCloud [38] have adopted microservice architectures to improve OLDI development and scalability [126]. These microservices are composed via standardized Remote Procedure Call (RPC) interfaces, such as Google’s Stubby and gRPC [15] or Facebook/Apache’s Thrift [42].