KNN-Q Learning Algorithm of Bitrate Adaptation for Video Streaming over HTTP

The increasing demands for seamless switching of high definition video streams have become the main driving force for the research of bitrate adaptive algorithms. Existing Q-learning based adaptive bitrate algorithm can obtain higher quality of experience (QoE) through dynamically learning the optimal behavior corresponding to the current network environment. However, ordinary Q-learning algorithm has the problem that it is difficult to encode continuous state values and the algorithm converges slowly in large state spaces. In this article, we combine the nearest neighbor (KNN) with Q-learning algorithms to proposes a KNN-Q learning algorithm that can deal with continuous state. Our experimental results show that in the bitrate adaptive scenario, the KNN-Q learning can achieve higher QoE and faster convergence speed than Q-learning algorithm.