Multi-Agent DDPG Based Resource Allocation in NOMA-Enabled Satellite IoT

Due to the scarcity of spectrum resources in Non-orthogonal Multiple Access (NOMA) systems and insufficient satellite-ground integration in satellite Internet of Things (IoT), this paper investigates its issue in spectrum resource management. We propose a resource allocation method based on Multi-Agent Deep Deterministic Policy Gradient (MADDPG) for NOMA enabled satellite IoT. We formulate the spectrum allocation problem of the satellite-ground integrated network as a distributed optimization problem. Then we decouple the problem into two sub-problems. Firstly, a user grouping method based on matching coefficients is defined, and a Linear Programming (LP) method is utilized for obtaining solution. Secondly, the power allocation problem is transformed into a multi-agent problem, where MADDPG is employed to allocate the power. Through this approach, the system is capable of real-time user association and spectrum resource allocation optimization, achieving optimal user grouping while maximizing system transmission rate. Based on the simulation results, the MADDPG-based method demonstrates fast convergence within 100 training iterations. The proposed MADDPG-based resource management method also achieves increased system transmission rate with more effective matching outcomes over Deep Deterministic Policy Gradient (DDPG), Orthogonal Multiple Access (OMA), and random allocation baselines.

I. Introduction

In Internet of Things (IoT), satellite communication systems have become a critical communication medium, which offers global coverage incorporating remote areas where traditional terrestrial networks fall short [1], [2]. Specifically, Low Earth Orbit (LEO) satellites are utilized for various IoT services such as IoT positioning, navigation, and environmental monitoring [3], [4], [5]. In the literature, several researches on channel allocation and spectrum utilization issues in satellite IoT are investigated. For example, Sanctis et al. [6] emphasized the importance of satellite communication systems for sensors and actuators in remote areas lacking terrestrial network connections. Obata and Takyu [7] explored interference detection techniques, assuming dynamic spectrum sharing between terrestrial and non-terrestrial communications. They propose a radio sensor to identify interference from ground systems with satellite earth stations. Hu et al. [8] proposed a solution for satellite IoT forward link multiplexing using high-order modulation to facilitate the transmission of different signals over the same frequency band simultaneously. However, higher-order modulation poses challenges for complex demodulation. Therefore, Yang et al. [9] designed a spectrum-sharing scheme based on soft frequency reassignment for satellite-ground integrated networks. Their slot allocation approach enhances spectral isolation and aims to optimize satellite spectrum utilization, albeit with increased interference. Although satellite communications have great potential, they are also facing a number of challenges, such as limited non-terrestrial spectrum resources, high transmission delays, and inter-satellite interference [10], [11].