Deep Reinforcement Learning Based Trajectory Design and Resource Allocation for UAV-Assisted Communications

In this letter, we investigate the Unmanned Aerial Vehicles (UAVs)-assisted communications in three dimensional (3-D) environment, where one UAV is deployed to serve multiple user equipments (UEs). The locations and quality of service (QoS) requirement of the UEs are varying and the flying time of the UAV is unknown which depends on the battery of the UAVs. To address the issue, a proximal policy optimization 2 (PPO2)-based deep reinforcement learning (DRL) algorithm is proposed, which can control the UAV in an online manner. Specifically, it can allow the UAV to adjust its speed, direction and altitude so as to minimize the serving time of the UAV while satisfying the QoS requirement of the UEs. Simulation results are provided to demonstrate the effectiveness of the proposed framework.

I. Introduction

Unmanned Aerial Vehicles (UAVs)-assisted communication is expected to play an important role in future wireless communications. There are many challenges that need to be addressed before UAVs can be effectively utilized for communication purposes [1]. The existing contributions on UAV-assisted communication systems can be divided into two categories: 1) UAV is statically deployed in the air to enhance wireless communication coverage; 2) UAVs are dynamically deployed, serving as the relay of the Base Station (BS) [2], and collecting Internet of Things (IoT) data, etc. Compared with the traditional ground BS, UAV-assisted-BS offeres the advantages such as enhanced mobility and increased likelihood of Line-of-Sight (LoS) communication with users. The existing contributions on dynamic deployment of UAV are normally based on: a) convex optimization algorithms [3]; and b) deep reinforcement learning (DRL) algorithms [4]. Compared with traditional convex optimization, DRL based algorithms may have higher performance and lower time complexity [5].