An Integrated Long Short-Term Memory Based Online Log Anomaly Detection

System logs record noteworthy information and become a valuable resource for tracking and investigating the status of a system. Detecting anomalies from logs as fast as possible can enhance quality of service. Although many deep learning algorithms have achieved good results in the field of log anomaly detection, we find that they assume that data is acquired and trained in bulk and may have high training time, while in reality log data streams are constantly arriving. Facing these challenges, this paper proposes a Long Short-Term Memory network (LSTM) based online log anomaly detection. We use transfer learning to train weak learners and integrate weak learners as strong learners for the constantly arriving log. We compare the performance of the methods using a public log dataset and experimentally validate the effectiveness of each module.

I. Introduction

System logs record noteworthy information and become a valuable resource for tracking and investigating the status of a system. Log anomaly detection has a wide range of application scenarios such as system operation and maintenance, early warning of the state of complex industrial systems.