Meta-DAMS: Delay-Aware Multipath Scheduler using Hybrid Meta Reinforcement Learning

The deployment of multipath transport protocols in the mobile environment can enhance the performance of delay-sensitive applications by enabling the simultaneous use of several network paths, resulting in faster data transmission. However, due to the heterogeneity of network paths, packets may not arrive on time or in order, affecting the performance of delay-sensitive applications. Therefore, a well-designed multipath scheduler is important to distribute data packets efficiently to guarantee the per-packet delay requirement. In this paper, we propose Meta-DAMS, a delay-aware learning-based multipath scheduler, aiming to ensure that end-to-end delay is below a predefined threshold for delay-sensitive applications. We introduce a hybrid meta reinforcement learning (meta-RL) architecture for Meta-DAMS in which offline meta-RL and online meta-RL are used to learn the optimal scheduling policy quickly and accurately in response to highly dynamic network conditions. Based on trace-driven emulation experiments, we demonstrate that Meta-DAMS surpasses state-of-the-art MP schedulers, ensuring a delay of 50 ms or less for 98% of packets after sufficient operational episodes, compared to the 83% achieved by existing MP schedulers. Even in initial operational episodes, Meta-DAMS maintains its superiority, guaranteeing 94% of packets with a delay of 50 ms or less, while the performance of the DQN-based MPQUIC scheduler drops to 72%. Meta-DAMS exhibits nearly triple the efficiency in terms of runtime compared to the DQN-based MPQUIC scheduler across varying episode numbers.