iCOS: A Deep Reinforcement Learning Scheme for Wireless-Charged MEC Networks

Computation offloading is an effective method in mobile edge computing (MEC) to relieve user equipment (UE) from the limited computation resource and battery capacity. Meanwhile, simultaneous wireless information and power transmission (SWIPT) can be applied to MEC to extend the operating time of the equipment. However, in multi-user network environment, diverse computation task requirements and changeable network channel states make it challenging to obtain offloading strategy timely and accurately. To address the issue, we propose an intelligent computation offloading scheme (iCOS) based on enhanced priority deep deterministic policy gradient (EPDDPG) algorithm to minimize the energy consumption of all the UEs by jointly optimizing the offloading decision, the central processing unit (CPU) frequency and the power split ratio in a dynamic SWIPT-MEC network. In particular, we improve the traditional fully-connected network structure to obtain both discrete and continuous action outputs, and accelerate neural network parameter updates by using prioritized experience tuples. Furthermore, we use dynamic voltage and frequency scaling (DVFS) technology to dynamically adjust the CPU frequency of local computing, and employ SWIPT technology to balance the charging and communication according to the obtained strategy. Simulation results show that the algorithm proposed in this paper can effectively reduce the energy cost of UEs, and complete more computation tasks within the delay limit.

I. Introduction

Recently, the expansion in Internet of Things and advancement in communication technology have paved a way of realizing user equipments intelligence, and the amount of data generated by these devices has shown an exponential growth trend. However, these terminal equipments (e.g., wearable devices, smart phones, tablet computers, etc.) are often resource-constrained and battery-powered, and may not be able to cope with the requirements of processing a large amount of raw data [1] timely and efficiently. Computation offloading can effectively alleviate the pressure of the terminal equipment by pushing the tasks from the resource-limited devices to the cloud, called mobile cloud computing (MCC) scheme, thereby reducing the energy consumption of the UE. However, these traditional MCC schemes based on remote public cloud may cause the long delay of performing data exchange through the network. Therefore, mobile edge computing (MEC) reduces the pressure of local data processing by taking advantage of resources at the edge of the networks, while reducing transmission delay [2]–[4].