DRL-Based Offloading for Computation Delay Minimization in Wireless-Powered Multi-Access Edge Computing

Wireless power transfer (WPT) and edge computing have been validated as effective ways to solve the energy-limited problem and computation-capacity-limited problem of wireless devices (WDs), respectively. This paper studies the wireless-powered multi-access edge computing (WP-MEC) network, where WDs conduct either local computing or task offloading for their individable computation tasks. We aim to minimize total computation delay (TCD) when each WD has a computation task to execute, referred to as the total computation delay minimization (TCDM) problem, by jointly optimizing the offloading-decision, WPT duration, and transmission durations of offloading WDs. The TCDM problem is a mixed integer programming (MIP) problem that is challenging to efficiently obtain the optimal or near-optimal solution. To tackle this challenge, we decompose the TCDM problem into the sub-problem of optimizing the WPT duration and transmission durations, and the top-problem of optimizing the offloading decision. For the nonconvex sub-problem, we design a worst-WD-adjusting (WDA) algorithm to efficiently obtain its optimal solution. For the top-problem, under the time-varying channel conditions, traditional optimization methods are hard to determine the optimal or near-optimal offloading decision within the channel coherence duration. To fast obtain the near-optimal offloading decision, we propose a deep neural networks (DNN)-based deep reinforcement learning (DRL) model, which takes the sub-problem solving as one component for utility evaluation. Finally, numerical results demonstrate that the proposed online DRL-based offloading algorithm achieves the near-minimal TCD with low computational complexity, and is suitable for the fast-fading WP-MEC network.

I. Introduction

Traditionally, powering wireless devices (WDs) by batteries is popular [1], [2], [3]. However, the capacity of batteries limits the lifetime of WDs, and narrows the application of WDs in practice. With the help of wireless power transfer (WPT) technology [4], [5], WDs are able to harvest energy from the radio frequency (RF) signal to charge the batteries, and extend the lifetime [6].