UAV-BS Trajectory Optimization Under Coverage, Backhaul and QoS Constraints Using Q-Learning

In this study, three-dimensional optimal trajectory planning is performed for the Unmanned Aerial Vehicle (UAV) with the base station (BS) attached to it to increase the service provided to the users. The case that heterogeneous quality of service (QoS) requirements for different users are considered. While planning the trajectory, the coverage area of the UAV-BS and backhaul capacity between the UAV-BS and Ground Base Station (GBS) are limited. Under these constraints, the aim is to find a trajectory for the UAV-BS that maximizes the total data rate provided to the users during the flight using reinforcement learning. With the application of Q-learning in our problem, the UAV-BS learns to take action to achieve the desired goal. As a result of trial and error processes with different learning parameters, appropriate parameters are determined and a reinforcement learning model is trained. Different communication scenarios are compared for analyzing the effects of the constraints. According to the effects of the mentioned constraints and heterogeneous QoS demands, UAV-BS’s trajectory preferences and total transmission rate changes are examined. Three prominent results shows the effects of coverage, backhaul, and heterogeneous QoS. The UAV-BS tends to increase its altitude as the coverage constraint increases. Moreover, the backhaul constraint forces the UAV-BS’s trajectory closer to the GBS. Lastly, UAV-BS takes into account different QoS requirements of users as much as possible. UAV-BS maximizes the total transmission rate by determining the most suitable trajectory to meet these constraints.