Meta-TFEN: A Multi-Modal Deep Learning Approach for Encrypted Malicious Traffic Detection

Malware poses a significant threat to internet security. Existing deep learning-based methods for malware traffic detection typically rely on single-modal features, overlooking the heterogeneity of encrypted traffic, thus limiting their detection performance. To address this limitation, this paper proposes a multi-modal deep learning approach called Meta-TFEN for detecting encrypted malicious traffic. The method utilizes TCN, Bi-GRU, and LSTM to extract multi-modal features including the payload of secure transport layer protocols, statistical features, and features of TLS encryption activities. It employs an fusion network to capture the dependencies between modalities and integrates discriminative features to enhance detection performance. Additionally, this paper uses a meta-learning framework for classification to enable rapid deployment of the model. The performance of Meta-TEEN evaluated on public datasets and its applicability in real-world environments is explored using real samples. The experimental results clearly demonstrate that the Meta-TEEN method surpasses other state-of-the-art methods in terms of accuracy.

I. Introduction

Detecting encrypted malicious traffic presents a formidable cybersecurity challenge in the digital domain. As Internet technology advances, a growing number of users are adopting encrypted communication protocols. Google's Transparency Report [1] published in 2022 revealed that the adoption of encrypted protocols has risen from 50% in 2014 to 95%. Furthermore, 97% of the top 100 global websites use Hypertext Transfer Protocol Secure (HTTPS). Although encryption technology can effectively protect user privacy, more and more attackers are beginning to use SSL/TLS encryption protocols to communicate with C&C servers in malicious software in order to evade detection. According to a report by Sophos Labs [2], most of the content detected in malicious TLS traffic comes from the initial malware infection, including loaders, payload deliverers, and document-based installers. Therefore, there is an urgent need to explore techniques for detecting encrypted traffic to protect user privacy, enhance network security defenses, identify and prevent malicious activities, and improve the security and reliability of networks.