Adaptive MARL-based Joint Cooperative Caching and Resource Allocation for Deep Edge Networks

With the rapid development of mobile communications services and applications, deep edge networks (DENs) have become a new paradigm for addressing data traffic and network delay by combining mobile edge computing (MEC) and artificial intelligence (AI). In this network, edge caching can provide low-latency and reliable services in resource-constrained scenarios by caching content in the wireless access points (APs). In this paper, we propose a joint cooperative caching replacement and resource allocation algorithm based on a multi-agent deep deterministic policy gradient (MADDPG), which aims to minimize the average content download delay. The proposed algorithm regards each AP as a reinforcement learning agent that receives diverse state information (e.g., user request behaviors, cache status updates, content popularity information, and network conditions) to learn the caching replacement and bandwidth resource allocation strategy to maximize system rewards. Simulation results show that the proposed algorithm can efficiently decrease global content download delay, improve network efficiency, and enhance the user experience when compared to the benchmark algorithm.

I. Introduction

The sixth-generation (6G) wireless communication network is expected to provide global coverage with improved spectrum, energy, cost efficiency, intelligence, and security [1]. ITU-T’s report [2] identifies the following three driving characteristics that are associated with the lifestyle and social changes of the next decade: 1) high-fidelity holographic society; 2) interconnectedness of everything; 3) engineering applications sensitive to latency. Thus, DENs provide a strong impetus to meet the requirements of future wireless communication networks, such as reinforcement learning [3].