Ensemble Methods for Anomaly Detection Based on System Log

Anomaly detection plays an important role in large-scale distributed systems. System logs are significant source of troubleshooting and problem diagnosis. Most of the existing anomaly detection methods apply only one machine learning model to extract feature from structured logs. However, each machine learning model has different strength towards different target system. It is hard for developers to know which is the best method to their practical problem at hand. This paper proposes two methods for anomaly detection based on the machine learning ensemble models. The first method takes mixture of experts to combine the weighted prediction of ensemble members to generate the final prediction. The second method divides the sample into n parts, then use n models to extracting features. Experimental results demonstrate the validity and accuracy of the proposed methods.