On Performance Evaluation of AI Models in Detecting Malicious Traffics

The power communication system has a vital impact on the stability, reliability, and security of power system operations. If adversaries gain access to the power communication network, they may obtain sensitive information or initiate attacks by malicious applications. To ensure the security of the power communication system, it is important to effectively identify possible security threats from malicious traffic. To this end, this paper proposes a packet bytes-based text Convolutional Neural Network (textCNN) AI model. In particular, the model can automatically extract the essential characteristics of various malicious traffic from massive network data. To demonstrate its performance, we evaluate the model with realistic open data set and compare it with several classical deep learning models. Extensive experiment results show that the proposed approach can identify malicious traffic with an accuracy of 99.84%, which is 0.57%, 1.48%, and 1.66% higher than that of LSTM, 1D-CNN, and 2D-CNN respectively. Moreover, regarding specific malicious applications identification, it can achieve the F1-score up to 99.39%, which is 5.21%, 12.84%, and 13.71% higher than that of LSTM, 1D-CNN, and 2D-CNN respectively.