Security is Readily to Interpret: Quantitative Feature Analysis for Botnet Encrypted Malicious Traffic

Nowadays, there are two problems in the network traffic analysis. One is that it’s difficult to detect the encrypted threat traffic resulting in the ineffective analysis and detection of attack characteristics of complex network such as botnet traffic. The other is that machine learning(ML) is featured by high false alarm rate, long training period and demanding data accuracy. In this work, the data packet was deeply analyzed through the data packet analysis method including the analysis of its header and content so as to realize the analysis of network traffic’s characteristics. The CTU-13 and MCFP data sets were dimension-reduced by the t-SNE algorithm, and the 2-dimensional space was mapped so that the distribution boundary of the data was observed and judged. Through eXtreme Gradient Boosting (XGBoost), Support Vector Machines (SVM), Random Forest, Logistic Regression and other machine learning algorithms, we detected the encrypted traffic and found that XGBoost and Random Forest were better for threatening dense flows. On the basis of feature analysis, we selected all, DNS, HTTP and deleted 20/30/40 importance features for ranking, and quantitatively explained the impact of each feature on threat dense flow classification. Through conducting the performed recursive feature elimination with XGBoost algorithm, this paper obtained the classification ranking and then concluded that the influence degree of overall characteristics was very uneven with the first six characteristics led by average of duration occupying a large weight. By obtaining the SHAP evaluation data of the selected all, DNS, HTTP, and deleted 20/30/40 importance features, this paper quantitatively explained the impact of each feature on the classification of threat dense traffic. In addition, this paper also carried out the impact verification of the importance characteristics. Because machine learning, including deep learning, is difficult to interpret. And to ensure the security of cyberspace, we make predictions all the time — to determine whether extensive encrypted traffic is malicious. It is important to interpret the quantitative influencing factors of the predictive model. This paper proposed the influence of features on the classification of encrypted malicious traffic by quantitative methods-SQEITF.