RAMPART: Reinforcement Against Malicious Penetration by Adversaries in Realistic Topologies

The increasing scale and complexity of networked computer systems, growing vulnerabilities and attack surfaces in the services and software running on these systems, and increasing reliance of mission critical workflows on such networked computer systems requires comprehensive cyber defense capabilities. However, manual monitoring and introspection often lacks the speed that these complex systems require for their defense. Therefore, it becomes crucial to design, develop, and train autonomous cybersecurity agents that can work with humans in defending the networked computer systems while sustaining the mission critical operational workflows. In this paper, we present a comprehensive framework for reinforcement learning (RL)-based cyber agent training called Reinforcement Against Malicious Penetration by Adversaries in Realistic Topologies (RAMPART). Using a model-driven architecture, RAMPART enables cybersecurity researchers to rapidly synthesize diverse scenarios, configurations and topologies (of networks, services, CVE-s, ports, workflows) on-demand, in a correct-by-construction manner, and with different levels of fidelity and abstraction, ranging from coarse-grained simulation, to hybrid simulation/emulation, to live networks. We envision that the configurability of the training environment and its instantiations at different levels of network simulation fidelity will allow RL approaches to scale and manage the complexity of the high-dimensional observation and action spaces.