Deep Reinforcement Learning Based Dynamic Time Slot Allocation in Unmanned Aerial Vehicle

With the continuous advancement of Industrial Internet of Things (IIoT) technology and the demand for ultra-high-speed wireless transmission, unmanned aerial vehicle (UAV) capable of effectively enhancing communication quality have become one of the primary applications for sixth generation (6G) wireless communications. However, UAV face challenges such as traffic burst and significant jitter when used for data transmission. Time-sensitive networking (TSN) has emerged to address real-time and deterministic transmission problems in UAV communications. However, existing scheduling algorithms struggle to adapt to the dynamic network changes and easily lead to scheduling failure and delay redundancy. This paper proposes a deep reinforcement learning (DRL) based dynamic time slot allocation (DTA) algorithm to optimize TSN scheduling in UAV communications. The key point of this approach is to use DRL to evaluate the historical information of scheduling result and predict subsequent decisions to ensure the completion of dynamic scheduling. The scheme firstly uses constraints to limit flows to complete scheduling, and dynamically allocates time slot lengths to flows with changing periods and lengths. It then uses the evaluation results to make path decisions and update the deep neural network (DNN) to complete the dynamic scheduling. When compared with other scheduling algorithms (No-DTA, Traverse-Edge, Link-Traverse-Edge), proposed scheme consistently outperforms them across various network scenarios. It achieves high scheduling success rates and maintains stable average scheduling delays for flows, making it a more effective scheduling solution.