Incorporating Distributed DRL Into Storage Resource Optimization of Space-Air-Ground Integrated Wireless Communication Network

Space-air-ground integrated network (SAGIN) is a new type of wireless network mode. The effective management of SAGIN resources is a prerequisite for high-reliability communication. However, the storage capacity of space-air network segment is extremely limited. The air servers also do not have sufficient storage resources to centrally accommodate the information uploaded by each edge server. So the problem of how to coordinate the storage resources of SAGIN has arisen. This paper proposes a SAGIN storage resource management algorithm based on distributed deep reinforcement learning (DRL). The resource management process is modeled as a Markov decision model. In each edge physical domain, we extract the network attributes represented by storage resources for the agent to build a training environment, so as to realize the distributed training. In addition, we propose a SAGIN resource management framework based on distributed DRL. Simulation results show that the agent has an ideal training effect. Compared with other algorithms, the resource allocation revenue and user request acceptance rate of the proposed algorithm are increased by about 18.15% and 8.35% respectively. Besides, the proposed algorithm has good flexibility in dealing with the changes of resource conditions.

I. Introduction

Artificial intelligence (AI) and wireless communication technology have rapidly spread in recent years. People all over the world have enjoyed the convenience brought by network services [1]. Especially in order to achieve the global coverage of wireless networks, a new type of space-air-ground integrated network (SAGIN) has been proposed to provide three-dimensional comprehensive connection services [2]. SAGIN needs to realize the seamless integration of space-based network, air-based network and ground-based network. Achieving ultra-reliable, low-latency wireless communication is an important goal of SAGIN. Satellites and unmanned aerial vehicles (UAVs) participate in network construction and provide services as air users or base stations (BSs) [3]. As an emerging wireless network architecture and new service paradigm, SAGIN can provide a reliable, stable and robust service supply for end users worldwide. A typical SAGIN scenario is shown in Fig. 1. The performance of SAGIN is largely dependent on 6G. SAGIN supported by 6G can bring benefits such as global coverage, ultra-low latency and high connection density, but 6G technology is not yet mature [4]. Especially for network operators, how to coordinate, manage and schedule the network resources of SAGIN is a severe challenge [5].