Deep Reinforcement Learning-Based Moving Target Defense for Multicast in Software-Defined Satellite Networks

The development of LEO satellite networks (LSN) makes them a potential solution to deliver broadcast/multicast traffic to deploy and upgrade massive amounts of Internet of Things (IoT) devices in future 6G networks. However, inherent resource constraints of LSN leave them vulnerable to a multitude of security threats, most notably distributed denial-of-service (DDoS) attacks. Existing solutions are primarily based on machine learning detection methods which are incapable of defending against unknown zero-day attacks. This paper presents an innovative solution leveraging deep reinforcement learning (DRL) to create a dynamic multicast tree based on moving target defense (MTD), aimed at enhancing the security of multicast services in LSN. The proposed solution adopts an adaptive orbital tree mutation (AOTM) scheme that dynamically adjusts multicast tree configurations considering quality of service (QoS) constraints to avoid attacks on vulnerable nodes. Simulations demonstrate the effectiveness of the AOTM scheme, showcasing its superior defense success rates compared to existing state-of-the-art algorithms.