Towards Continuous Cyber Testing with Reinforcement Learning for Whole Campaign Emulation

Modern automated penetration testing uses rule-based procedures and model-checking concepts to search through all possible attacks on network models and identify those that violate some correctness or security property by generating an attack graph. By generating all possible attacks, modern, top-down approaches inherently do not isolate the few attacks that matter the most. This weakness is exacerbated in future network settings like 5G and Internet of Things (IoT) settings where networks are expected to have thousands of hosts (or more) and evolve over time. This has created a perception that the attack graph concept itself is inadequate, in turn hindering the automation of cyber testing. Recent research re-positions automated attack graph generation as a best practice in cyber defense by applying deep reinforcement learning (RL). While recent research into penetration testing with RL has seen a rapid growth in interest, a clear concept of operational use has not been defined. We define and provide formalism for the concept of whole campaign emulation (WCE). We present WCE as both a challenge problem and a framework for automating cyber T&E with RL. This manuscript captures an RL-oriented perspective on the past, present, and future of attack graph generation, and serves as a primer from researchers and practitioners alike. With WCE, organizations from small businesses to nation-states can feasibly institute continuous cyber T&E with low test costs and low disruption to operations.