Stigmergy and Hierarchical Learning for Routing Optimization in Multi-Domain Collaborative Satellite Networks

The integration of Software-Defined Networking (SDN) and Artificial Intelligence (AI) presents promising opportunities for managing and optimizing LEO satellite network routing. However, as the scale and coverage of satellite networks continue to expand, challenges are posed to both centralized and distributed architectures in terms of managing network information and coping with routing complexity. To overcome these challenges, leveraging distributed SDN technology, a stigmergy multi-agent hierarchical deep reinforcement learning routing algorithm is proposed in multi-domain collaborative satellite networks. A pheromone-based mechanism is incorporated to facilitate collaboration during independent training, and hierarchical control is employed to decouple the complexity of cross-domain routing decisions. Simulation results demonstrate that our proposed algorithm exhibits good scalability and performance in large-scale satellite networks.

I. Introduction

With the ongoing advancements in 5G technology, satellite networks are increasingly recognized as a critical element of future communication infrastructures [1], [2], [3]. This could significantly expedite the growth of the diverse Internet of Things (IoT), including smart wearables, industrial IoT, and other connected devices, all benefiting from enhanced communication capabilities [4], [5]. The recent launch of new low Earth orbit (LEO) satellite internet projects by innovative companies such as SpaceX and OneWeb has reignited interest in the promising vision of seamless global communication through LEO satellites [6]. As LEO satellite networks become more integral to the communication infrastructure, routing optimization emerges as a fundamental challenge in ensuring their efficiency and effectiveness [7]. The dynamic nature of LEO satellite networks leads to the constant change in their topological connectivity over time, necessitating frequent maintenance and updates to the routing tables of each node [8]. As the scale of LEO satellite networks expands, obtaining real-time network status information becomes a challenge [9]. Excessive flooding transmissions result in substantial overhead for network state monitoring. Moreover, the increased scale of the network adds complexity to routing algorithms, posing challenges to their scalability and real-time performance.