MALT: Fine-Grained Microservice Profiling for Request Latency Anomaly Localization

Recent years have witnessed the shift of cloud datacenter architecture from monolithic to microservice. In microservice, request latency which fundamentally determines application performance, has become a crucial concern for cloud operators. Due to various issues with hosts, networks, and remote services, microservices can encounter anomalies when processing requests, causing sudden latency spikes and potential Service- Level Agreement (SLA) violations. Existing performance monitoring tools lack the ability of collecting performance data containing request-level semantics under low overhead. They also do not address the diverse threading models of microservices, posing challenges in locating Request Latency Anomalies (RLAs). This paper proposes MALT, a performance profiling tool that collects fine-grained, request-level data and locates RLAs in microservices. MALT hooks the critical functions when microser-vices process requests and extracts request-level semantic data from function parameters or packet payloads. To characterize the complex and diverse microservice threading behaviors, MALT maps the collected data to each request according to request-level semantics and models the request execution process of microservices as a Directed Acyclic Graph (DAG), from which the path with longest latency is extracted to locate RLAs. We apply MALT to an open-source microservice in an experimental network and a microservice deployed in a production network. Results show that MALT achieves an average precision of 93% in locating four types of RLAs. These results further guide the optimization of request latency in production microservice, where the average request latency is reduced by up to 41.8 % by the heuristic methods.

I. Introduction

In recent years, the number of applications adopting mi-croservice architectures in cloud datacenters has proliferated rapidly [1]–[3]. Compared to monolithic architectures where all components of an application are tightly coupled, microser-vice architectures decompose applications into loosely coupled services. This enables advantages like cross-team development and friendly deployment. Microservices interconnect via re-mote procedure calls (RPCs), HTTP, or other methods. The time that a microservice takes to execute an RPC or HTTP request, noted as request latency, directly impacts application performance, and thus becomes the most critical performance metric for datacenter operators. When micro service request latency spikes, operators must leverage application and system performance data to pinpoint anomalies.