Network Traffic Anomaly Detection Using Quantile Regression with Tolerance

Network traffic anomaly detection describes a time series anomaly detection problem where a sudden increase or decrease (called spikes) in network traffic is predicted. Data is modeled with the trend and heteroscedastic noise component. Traditional autoregressive models struggle to capture data changes effectively, making anomaly detection difficult. Our approach is to generate upper and lower limits by using quantile regression. We use a deep learning based multilayer perceptron model to predict five data quantiles 1, 25, 50, 75, and 99. The upper and lower limits are calculated as differences between the quantile-1 and quantile-99. Any data that is outside these limits are considered as an anomaly. We also add tolerance to these limits to add flexibility to anomaly detection. Anomalies and non-anomalies are labeled to get a binary classification task. Anomaly detection is class imbalanced by nature; therefore, precision, recall, and F-1 score are computed to evaluate the proposed anomaly detection method. We conclude that choosing tolerance is a tradeoff between false alarms and missing anomaly detections.