Deep Reinforcement Learning Based 3D-Trajectory Design and Task Offloading in UAV-Enabled MEC System

In unmanned aerial vehicle (UAV) enabled mobile edge computing (MEC) system, rotary-wing UAV can be dispatched to fly close to ground terminals (GTs) to execute their offloaded tasks. This can extend GTs' computing capability and save their energy cost. However, to enhance the energy efficiency of UAV propulsion, ensure successful completion of each GT's mission, and maintain a stable UAV-GT uplinks, it is crucial to design a rational UAV 3D trajectory and mission offloading strategy. To address this joint problem, We further build a deep reinforcement learning (DRL)-based framework in the UAV-enabled MEC system, which applies Double Deep Q-learning (D-DQN) and convex optimization techniques to jointly work out the near-optimal UAV 3D-trajectory and task offloading strategy among GTs in an online manner. With the proposed framework, the UAV-enabled MEC system can finish each GT task quickly and save the UAV mission time and propulsion energy effectively. Numerical results show that the DLR-based framework outperforms the benchmark solutions, e.g., In scenario 1, the throughput, the average data transferring rate through UAV-GT links, and the average energy efficiency of UAV compared to DQN are improved from 64.24Kb to 72.99Kb, 4.72Kbps to 20.75Kbps, 0.32bits/J to 0.36bits/J, respectively, and almost the same as the existing Travelling Salesman Problem (TSP) solution, making the energy efficient edge computing truly viable in UAV-enabled MEC system.

I. Introduction

Currently, wireless communications supported by low-altitude unmanned aerial vehicles (UAVs) are appealing and expected to be enormous. These UAV-aided wireless networks can be swiftly deployed because of the high flexility and mobility of the UAV [1]. UAV also can be deployed to work as the mobile edge computing (MEC) server in the considered wireless networks to execute the tasks offloaded from ground terminals (GTs) [2], [3], [4]. Such UAV can fly close to the GTs to facilitate the tasks offloading. Then, the computation capability and the battery lifetime of GTs can be effectively improved and extended [5], [6]. The UAV-enabled MEC also can be practically supported by cache to have the most frequently offloaded tasks pre-stored in its local memory to avoid redundant data transmission from GT to UAV during task offloading [7], [8].