Real-Time Analysis of Encrypted DNS Traffic for Threat Detection

Domain Name System (DNS) tunneling is a well-known cyber-attack that allows data exfiltration - the attackers exploit this tunnel to extract sensitive information from the system. Advanced Persistent Threat (APT) attackers encapsulate malicious traffic in a DNS connection to elude security mechanisms such as Intrusion Detection System (IDS). Although different techniques have been implemented to detect these targeted attacks, their rise induces a threat to Cyber-Physical Systems (CPS). The DNS over HTTPS (DoH) tunnel detection is a challenge because the encrypted data prevents an analysis of DNS traffic content. In this paper, we present a novel detection system that identifies malicious DoH tunnels in real time. We study the normal traffic pattern and based on that, we define a profile. The objective of this system is to detect malicious activity on the system as early as possible through a lightweight packet by packet analysis based on a real-time IDS classifier. This system is evaluated on three available data sets and the results obtained are compared with a machine learning technique. We demonstrate that the identification of anomalous activity, in particular DoH tunnels, is possible by analyzing different traffic features.