Intelligent IDS Chaining for Network Attack Mitigation in SDN

Recently emerging software-defined networking allows for centralized control of the network behavior enabling quick reactions to security threats, granular traffic filtering, and dynamic security policies deployment making it the most promising solution for today’s networking security challenges. Software-defined networking coupled with network function virtualization extends conventional security mechanisms such as authentication and authorization, traffic filtering and firewalls, encryption protocols and anomaly-based detection with traffic isolation, centralized visibility, dynamic flow control, host and routing obfuscation, and security network programmability. Virtualized security network functions may have different effects on security benefit and service quality, thus, their composition has a great impact on performance variance. In this study, we focus on solving the problem of optimal security function chaining with the help of reinforcement machine learning. In particular, we design an intelligent defense system as a reinforcement learning agent which observes the current network state and mitigates the threat by redirecting network traffic flows and reconfiguring virtual security appliances. Furthermore, we test the resulting system prototype against a couple of network attack classes using realistic network traffic datasets.