Machine Learning-based Approach to Detect Online Performance Problems in Distributed Systems

Detecting the root cause of a performance problem in a distributed system is a complex and costly task. Identifying the fault, which can be internal or external, requires a deep knowledge of the system, and tools that allow processing and filtering large amounts of information. This paper describes a methodology to identify performance problems on distributed systems. The operation flows are inferred from log files and are used to measure a performance indicator. This information is complemented by system data, such as metrics of each node (CPU, memory, disk), that is analyzed with two machine learning techniques, multivariate regression and one class support vector machine (OCSVM), with the purpose of predicting the expected performance and the presence of unusual events. The mentioned model is implemented in a tool called Logmapper, that is used to validate the approach in a controlled test environment, where several types of failures that affect performance were applied. The validation results are composed of the validation metrics of the learning methods and the response given by the tool when failures were induced in the distributed system.