DOFMS: DRL-Based Out-of-Order Friendly Multipath Scheduling in Mobile Heterogeneous Networks

Multipath transmission brings strong bandwidth aggregation capability for services in wireless networks. Nonetheless, the heterogeneous nature of paths and the motion of terminals results in varying transmission delays, leading to out-of-order (OFO) delivery and transmission quality decrease. Traditional algorithms, due to their solid operating logic, fail to strike a balance between high bandwidth and low OFO extent. Recent studies have focused on utilizing learning algorithms to find a multi-metric joint optimal transmission strategy. In light of this, this paper proposes a framework called DRL-based OFO-Friendly Multipath Scheduling (DOFMS) to ensure high bandwidth and low OFO extent transmission in mobile heterogeneous networks. In particular, the framework introduces a novel OFO evaluation index to assess the degree of OFO more accurately. To achieve elastic scheduling, the framework employs the Double Deep Q Network (DDQN) to dynamically regulate the scheduling ratio. Recognizing the dynamic and unpredictable nature of path delays, an asynchronous module is introduced to enhance learning accuracy. Experimental results demonstrate that the DOFMS reduces the OFO rate by 25% compared to traditional bandwidth aggregation algorithms, while maintaining low bandwidth and packet loss rates. Furthermore, compared to conventional OFO avoidance algorithms, the DOFMS improves bandwidth by 4% and reduces fluctuation by 90%.