Trajectory Design and Generalization for UAV Enabled Networks:A Deep Reinforcement Learning Approach

In this paper, an unmanned aerial vehicle (UAV) flies as a base station (BS) to provide wireless communication service. We propose two algorithms for designing the trajectory of the UAV and analyze the impact of different training approaches on transferring to new environments. When the UAV is used to track users that move along some specific paths, we propose a proximal policy optimization (PPO) -based algorithm to maximize the instantaneous sum rate (MSR-PPO). The UAV is modeled as a deep reinforcement learning (DRL) agent to learn how to move by interacting with the environment. When the UAV serves users along unknown paths for emergencies, we propose a random training proximal policy optimization (RT-PPO) algorithm which can transfer the pre-trained model to new tasks to achieve quick deployment. Unlike classical DRL algorithms that the agent is trained on the same task to learn its actions, RT-PPO randomizes the features of tasks to get the ability to transfer to new tasks. Numerical results reveal that MSR-PPO achieves a remarkable improvement and RT-PPO shows an effective generalization performance.

I. Introduction

UAVs have been viewed as a critical role in future wireless communication systems because of their mobility and flexibility. UAVs can serve as aerial base stations (BS) [1] or relay nodes [2], [3] to help improving the performance of the ground wireless networks in various scenarios, such as temporary crowed areas and disaster areas. The links between UAVs and ground users have a higher probability to be Line-of-Sight (LoS) because of the high altitude of UAVs [1]. However, there are many challenges among which the optimal deployment of UAVs is an important problem.