Deep Reinforcement Learning Based Trajectory Design for Customized UAV-Aided NOMA Data Collection

In this letter, we design a customized channel model based on ray tracing (RT) and machine learning (ML). RT is used to generate path loss for selected areas. The generated path loss is trained through the Deep Neural Network (DNN). The channel model can output the path loss by inputting the transceiver’s three-dimensional (3D) coordinates. We investigated the task of collecting data by unmanned aerial vehicle (UAV) based on the customized channel model. The time it takes for the UAV to finish collecting data generated by ground user equipment (UE) is minimized. We combine non-orthogonal multiple access (NOMA) to analyze UAVs’ optimal 3D and 2D flight trajectories and demonstrate that 3D outperforms 2D. The optimized proximal policy optimization (optimized PPO) based deep reinforcement learning (DRL) algorithm is proposed to address this issue. The UAV can adjust its speed and direction. Simulation results demonstrate the effectiveness of the proposed customized channel model and algorithm.

I. Introduction

Unmanned aerial vehicle (UAV) has been widely used in various communication scenarios due to its advantages, such as high mobility, low cost, and line-of-sight (LOS) propagation. UAV has three typical applications in traditional wireless communications: 1) UAV-aided ubiquitous coverage; 2) UAV-aided relaying; 3) UAV-aided information dissemination and collection [1].