TDLens: Toward an empirical evaluation of provenance graph-based approach to cyber threat detection

To combat increasingly sophisticated cyber attacks, the security community has proposed and deployed a large body of threat detection approaches to discover malicious behaviors on host systems and attack payloads in network traffic. Several studies have begun to focus on threat detection methods based on provenance data of host-level event tracing. On the other side, with the significant development of big data and artificial intelligence technologies, large-scale graph computing has been widely used. To this end, kinds of research try to bridge the gap between threat detection based on host log provenance data and graph algorithm, and propose the threat detection algorithm based on system provenance graph. These approaches usually generate the system provenance graph via tagging and tracking of system events, and then leverage the characteristics of the graph to conduct threat detection and attack investigation. For the purpose of deeply understanding the correctness, effectiveness, and efficiency of different graph-based threat detection algorithms, we pay attention to mainstream threat detection methods based on provenance graphs. We select and implement 5 state-of-the-art threat detection approaches among a large number of studies as evaluation objects for further analysis. To this end, we collect about 40GB of host-level raw log data in a real-world IT environment, and simulate 6 types of cyber attack scenarios in an isolated environment for malicious provenance data to build our evaluation datasets. The crosswise comparison and longitudinal assessment interpret in detail these detection approaches can detect which attack scenarios well and why. Our empirical evaluation provides a solid foundation for the improvement direction of the threat detection approach.