APEX: Characterizing Attack Behaviors from Network Anomalies

Networks regularly face various threats and attacks that manifest in their communication traffic. Recent works proposed unsupervised approaches, e.g., using a variational autoencoder, that are not only effective in detecting anomalies in network traffic, but also practical as they do not require ground truth or labeled data. However, the problem of characterizing anomalies into different attack behaviors is still less explored; in this work, we study this specific problem. We develop APEX, a framework that employs data mining approaches in a semisupervised way to extract the attack patterns from anomalous traffic and links them to specific attack types. APEX comprises two levels of mining: the first level extracts patterns in anomalous network flows, and the second level characterizes behaviors in the extracted patterns into different attack classes. We carry out extensive experiments on real network traces obtained from the MAWI traffic archive. The evaluations demonstrate that APEX is effective in extracting distinguishable behaviors of network attacks from anomalous traffic, and provide useful insights to security analysts investigating the anomalies.