Deep Reinforcement Learning Empowers Wireless Powered Mobile Edge Computing: Towards Energy-Aware Online Offloading

Deep integration of wireless power transmission and mobile edge computing (MEC) promotes wireless powered MEC to become a new research hotspot in the field of Internet of Things. In this paper, we focus on the joint optimization problem of online offloading decision and charging resource allocation for minimizing task accomplishing time in dynamic time-varying wireless channel scenarios. The optimal solution involves addressing a mixed integer programming problem in real time, which is proved to be NP-hard, and imposes nontrivial challenges to design with conventional optimization methods. To efficiently address this problem, we leverage the deep reinforcement learning (DRL) technology to propose an energy-aware online offloading algorithm called EAOO. EAOO algorithm learns empirically the online offloading decision policies via a well-designed DRL framework, and adopts the feasible solution region analysis method to implement the charging resource allocation. We further propose a novel feasible decision vector generation method, and incorporate the crossover and mutation technology to expand the offloading vector search space with the provable feasibility guarantee. Extensive experimental results show that, our EAOO algorithm outperforms existing baseline algorithms, and achieves near-optimal performance with low CPU execution latency, which well satisfies the practical requirements of real-time and efficiency.

I. Introduction

With the rapid development of the sensing and communication technology, the Internet of Things (IoT) shows wide applications in numerous fields. In practice, the amount of data generated by IoT equipments is increasing massively [1]. If the data is processed by cloud computing, endless requirements for spectrum resources, transmission bandwidths and data processing capacities will inevitably cause the cloud center to be overwhelmed. Recent years have witnessed the mobile edge computing (MEC) technology as a promising technology to relieve the heavy pressure of the cloud center [2], [3], [4], [5], [6], [7], [8]. MEC technology can efficiently improve the task accomplishing time and energy efficiency [9], [10], [11], [12], [13] by offloading computation-intensive tasks of IoT devices to near edge servers with powerful computing abilities. In particular, affected by dynamic wireless communication conditions and server resources, the task of a device follows two computation modes: local computing and edge computing. Hence, enormous efforts have been devoted to tackle the binary offloading decision and resource allocation problems of MEC in the practical applications.