Deep Reinforcement Learning for Communication and Computing Resource Allocation in RIS Aided MEC Networks

In this paper, we apply reconfigurable intelligent surface (RIS) technique to aid the computation offloading of mobile edge computing (MEC) network and investigate how it can be exploited to reduce computation offloading delay. In order to minimize the long-term computation offloading delay, we formulate an optimization problem, which jointly optimizes the power control, computation offloading volume, the edge computing resource assigned to each user equipment (UE), as well as the RIS phase shift. To tackle this problem, we first convert it into a Markov decision process (MDP), then propose an efficient algorithm based on deep reinforcement learning (DRL), namely deep deterministic policy gradient (DDPG). Numerical results demonstrate that 1) compared to the MEC network without RIS, the RIS aid MEC network can achieve lower delay; 2) the proposed DDPG-based scheme can learn from the RIS aided environment to effectively reduce the computation offloading delay.

I. Introduction

FOR the Internet of Things (IoT) field with vigorous development, the proliferation of sensors, electronics, and machines are expected to realize interconnection via the Internet [1]. With the help of IoT technology, the physical world in diverse scenarios, such as smart railway and industrial automation can be monitored and controlled through processing and analyzing the collected data [2]. In the IoT network, myriads of equipments may generate a lot of compute-intensive and delay-sensitive tasks. With the purpose of emancipating the resource-limited equipments and supplying low-latency computing services, mobile edge computing (MEC) can take advantage of computing capability at the network edge [3]. As a result, the equipments' computation-intensive tasks are able to be offloaded to the edge nodes, such as access points (APs), and carried out at the AP, which prevents the happening of the remote transmission to the cloud [4]. However, in the harsh environments, such as high-speed railway (HSR) network [5] and cell edge, the offloading rate will decrease remarkably, which will prolong the offloading delay and the overall delay of carrying out the computation-intensive task. In this case, the performance of MEC networks is greatly limited. Consequently, it is crucial to promote the performance of MEC networks from a communications perspective.