DeepRS: Deep-Learning Based Network-Adaptive FEC for Real-Time Video Communications

As real-time multimedia streaming thriving, Forward Error Correction (FEC) methods have been studied and applied extensively these years. Most of researchers paid their attention to the coding algorithms, attempted to balance the trade off between recovery ratio and delay with fewer redundance. However, when packet loss pattern changes dynamically, the redundance waste is too serious to be ignored. In this work, we propose a novel algorithm which adjusts the redundance ratio of FEC encoder according to the prediction of packet loss. Receivers are additionally required to feedback observed packet loss pattern. Streaming sender collects the feedbacked packet loss pattern and predicts the number of packet loss in the incoming short period. As for implementation, we adopt long short-term memory (LSTM) network as our deep learning algorithm, and exquisitely embed it in our adaptive FEC system. With the extensive experiments, our proposed scheme outperforms other FEC methods greatly both in the simulations and evaluations on traces observed from the real world.