Federated Reinforcement Learning with Constellation Collaboration for Dynamic Laser Inter-Satellite Link Scheduling

The scheduling of the laser inter-satellite links (LISLs) can effectively improve the network performance but is challenging in mega-constellations due to the large amount of satellites. In this paper, inter-OP LISL scheduling is realized by federated reinforcement learning, which reduces complexity by decomposing global scheduling into independent decisions for each satellite and solved by multi-agent reinforcement learning. Based on the constellations collaboration, federated learning (FL) with partial and global aggregations is utilized to reduce the overhead for the model update, where the geostationary orbit (GEO)-GEO collaboration for global aggregation can be performed infrequently since the indirect associations among the GEO satellites according to the motion of low Earth orbit (LEO) satellite. Meanwhile, simulation results indicate that the proposed method reduces the average hop count by over two hops and decreases the number of LISLs by over 25% compared to the fixed LISLs, and the model update overhead is reduced to 46.6% of that in the centralized method.