Log Based Anomaly Detection: Relation Between The Logs

For large-scale software-intensive systems, high availability and reliability are critical. Given that these systems offer numerous services to users, even a minor issue with them could cause user annoyance and possibly severe financial losses. As a result, detecting anomalies in software-intensive systems is crucial. As attackers become more skilled, the conventional log anomaly detection method is no longer effective. Modern enterprise IT systems generate many log data to monitor system status, potential errors, and performance metrics. As these systems become more complex, manual analysis of log data becomes more challenging. Several critical aspects seem to have been overlooked by the existing work. The experimental datasets, evaluation metrics, and settings are related to these aspects. We will explore this problem in depth using deep learning by parsing the logs and extracting the valuable information inside the log that could help researchers, auditors and software developers maintain their software. The proposed method should achieve higher performance than other log anomaly detection methods.