RCP: A Reinforcement Learning-Based Retransmission Control Protocol for Delivery and Latency Sensitive Applications

This paper presents the design and performance evaluation of a new machine learning-based transport protocol called Reinforcement learning-based retransmission Control Protocol (RCP). Unlike prior transport protocols that aim at either providing guaranteed end-to-end delivery, e.g., TCP, or minimizing the end-to-end delay, e.g., UDP, RCP aims to optimize any achievable combination of these objectives, as specified by an application layer utility function. The utility function in this paper can be quite general, and can capture a combination of delay and packet delivery metrics. RCP can be thought of as an intelligent middle-ground between UDP and TCP, that maps application layer objectives subject to what is learned about the network state. It is window-based like TCP, but retransmissions are decided based on a reinforcement learning algorithm to maximize the application layer utility function. RCP employs a reinforcement learning method, double Q-Learning network, to learn the best strategy in real-time from a history of packet transmission/re-transmission experiences. No assumption on the shape of the utility function is needed. RCP is evaluated under a wide range of network settings, and is found to outperform UDP, TCP, and ARQ for almost all settings. The performance is also evaluated with respect to TCP-friendliness and network stability.