DRL-based Underlay Dynamic Spectrum Access for Cognitive Satellite Networks under Spectrum Sensing Errors

This paper investigates dynamic spectrum access (DSA) for cognitive satellite networks (CSNs), where a non-geostationary orbit (NGSO) satellite acting as a secondary user (SU) shares a segment of spectrum licensed to the primary user (PU) geostationary (GSO) satellite system. Considering the influence of spectrum sensing errors on spectrum sharing, a new problem about joint channel selection and power control is formulated as a sequential decision-making process, to maximize a long-term throughput of the NGSO under an interference constraint for the GSO system. Due to the imperfect spectrum information, we employ deep reinforcement learning (DRL) with a double deep Q-learning neural network to solve the problem by learning the set of DSA policies with stabilized convergence. We consider three cases to evaluate the performance of the proposed algorithm and the simulation results show that the throughput and the average transmission power of NGSO can converge quickly in a dynamic spectrum environment. Besides, the spectrum utilization can be maximized with meeting the transmission power constraint proposed by GSO for NGSO.