A Joint Trajectory and Computation Offloading Scheme for UAV-MEC Networks via Multi-Agent Deep Reinforcement Learning

Unmanned Aerial Vehicle (UAV)-assisted mobile edge computing (MEC) has emerged as a promising solution to support the computation-intensive tasks in the Internet of Things (IoT) networks. As for the operation of UAV-assisted MEC, jointly design of the UAV trajectory control and computation offloading strategies becomes the key for achieving high offloading efficiency, which is extremely challenging due to the uncertain and dynamic demands in the network. In this paper, aiming at maximizing the offloading task amount, we propose an Multi-Agent joint TrAjectory and Computation Offloading (MA-TACO) scheme, where all related factors including task type variety, quality of service (QoS) guarantee, and service fairness are taken into account. To facilitate each UAV to obtain the best joint strategy under dynamic network environment, considering the complex decisions with both continuous and discrete variables, we develop an Optimization-oriented Multi-Agent Deep Reinforcement Learning approach (OMADRL), where each UAV could autonomously learn the trajectory decision to adapt to the dynamic demands, and the offloading decision would be made by solving a mixed-integer programming problem based on the observations, which would be utilized to guide the trajectory learning. Comparing with solely relying on learning, such an optimization-oriented way could reduce the action space dimension and make each UAV achieve the best strategy faster. The simulation results indicate the effectiveness of the proposed scheme.

I. Introduction

With the recent arrival of the IoT era, many computation-intensive applications have emerged, such as autonomous driving, virtual reality, and facial recognition, etc, which require substantial computation and energy resources to support. In general, light-weight devices have limited computing capability and battery capacity. Hence, these computation-expensive tasks are hardly handled by such devices due to their resource constraints [1]. In this context, the concept of mobile edge computing (MEC) has been proposed [2] [3]. By deploying servers at the edge of the network, devices can offload their computation tasks to nearby MEC servers while receiving the results with low latency and low energy consumption [4]–[8].