DeepSCC: Deep Learning-Based Fast Prediction Network for Screen Content Coding

Screen content coding (SCC) is an extension of high efficiency video coding (HEVC), and it is developed to improve the coding efficiency of screen content videos by adopting two new coding modes: Intra Block Copy (IBC) and Palette (PLT). However, the flexible quadtree-based coding tree unit (CTU) partitioning structure and various mode candidates make the fast algorithms of the SCC extremely challenging. To efficiently reduce the computational complexity of SCC, we propose a deep learning-based fast prediction network DeepSCC that contains two parts: DeepSCC-I and DeepSCC-II. Before feeding to DeepSCC, incoming coding units (CUs) are divided into two categories: dynamic CTUs and stationary CTUs. For dynamic CTUs having different content as their collocated CTUs, DeepSCC-I takes raw sample values as the input to make fast predictions. For stationary CTUs having the same content as their collocated CTUs, DeepSCC-II additionally utilizes the optimal mode maps of the stationary CTU to further reduce the computational complexity. Compared with the HEVC-SCC reference software SCM-8.3, the proposed DeepSCC reduces the encoding time by 48.81% on average with a negligible Bjøntegaard delta bitrate increase of 1.18% under all-intra configuration.

I. Introduction

Screen content video refers to video captured from the display screen of an electronic device, and it has been applied to many screen sharing based applications, such as online education, remote desktop, and web conferencing [1]. Besides the traditional camera-captured natural image blocks (NIBs), screen content videos contain a significant amount of stationary or dynamic computer-generated screen content blocks (SCBs). Compared with NIBs, SCBs exhibit different characteristics, including no sensor noise, large flat areas with a single color, repeated patterns in the same frame and limited colors. Leveraging on these special characteristics of screen content videos, the Joint Collaborative Team on Video Coding (JCT-VC) has developed Screen Content Coding (SCC) extension [2] on top of High Efficiency Video Coding (HEVC) [3], and it outperforms HEVC by achieving over 50% Bjøntegaard delta bitrate (BDBR) [4] reduction for typical screen content videos.