DREVAN: Deep Reinforcement Learning-based Vulnerability-Aware Network Adaptations for Resilient Networks

In this work, we proposed a vulnerability-aware network adaptation framework that can generate a robust network topology against epidemic attacks by leveraging deep reinforcement learning (DRL) to build a resilient network. We call our proposed framework DREVAN, representing Deep REinforcement Learning-based Vulnerability-Aware Network Adaptations. The goal of the proposed DREVAN is to minimize security vulnerability caused by epidemic attacks exploiting the software monoculture for node compromise while maximizing network connectivity in terms of the size of the giant component. To be specific, the DREVAN aims to autonomously identify a pair of network adaptation budgets for adding or removing edges (i.e., how many edges to add or remove) by leveraging DRL. In this work, we tackle the inherent challenge of using DRL in reducing the learning curve of a DRL agent by proposing two algorithms. First, we proposed a vulnerability ranking algorithm of edges and nodes, namely VREN, for the DRL agent to select which edges to add or remove based on the lowest expected vulnerability between two nodes or the highest vulnerability of edges, respectively. Second, we also developed a Fractal-based Solution Search algorithm (FSS) to effectively direct the DRL agent towards the effective samples to visit and quickly identify and converge to an optimal solution (i.e., budget sizes of edge additions and removals). Via our extensive comparative performance analysis of the six different schemes, we demonstrated the outperformance of our proposed DREVAN-based schemes over counterpart and baseline schemes.