TRCC: Transferable Congestion Control With Reinforcement Learning

The breathtaking progress in machine learning has motivated studies in learning-based congestion control algorithms, which are expected to adjust congestion window choices according to the dynamic network environment. Nevertheless, existing learning-based congestion control protocols are mostly designed and trained for a specific network environment. When applied to a different network environment, the previously trained model may see a considerable degradation in performance. As the rapid development of communication technologies has given rise to the emergence of a diversity of new networks, it is desirable for learning-based congestion control models to quickly transfer to different network environments. Driven by this motivation, we propose a novel transferable congestion control (TRCC) protocol, which takes full advantage of both reinforcement learning (RL) and transfer learning to intelligently cope with network congestion in scenarios. The key idea to enable a fast transfer from the source network environment to the target network environment is to fine-tune a well-trained model in the source network to suit the target network in a time-effective way. We theoretically prove the transferability and quick convergence of our proposed transfer RL-based congestion control algorithm by deriving the Markov transition matrix and the similarity of the reward function. Our experiments validate that TRCC can converge in a new network environment in a short time while achieving comparable performance with baseline algorithms.