Neural Network-Based Error Concealment For VVC

In this paper we introduce an error concealment method for VVC based on deep recurrent neural networks, which employs the PredNet model to estimate missing video frames by using past decoded frames. The network is trained using the BVI-DVC data set to infer even full-HD frames. We integrated our proposed model in the VVC reference software VTM for its evaluation. It performs, in average, 6 dB or up to 5 dB better than the frame copy model in terms of PSNR measurements for a concealed I-frame or P-frame, respectively.

1. Introduction

The evolution of technologies for the display and recording of video signals has been responding to the rising demand for higher resolution devices. To meet these demands, the state-of-the-art video coding standard Versatile Video Coding (VVC) [1] has been released in summer 2020. This is driving all types of communication systems to increase their capacity of conveying information, from which video contents are predominant. For instance, by 2022 nearly four-fifths of the world’s mobile data traffic will be video [2]. Many applications like video surveillance, tele-medicine and smart car navigation systems require greater resolution and lower latency video communication systems [3].