Machine Learning-Based Fast Intra Mode Decision for HEVC Screen Content Coding via Decision Trees

The screen content coding (SCC) extension of high efficiency video coding (HEVC) improves coding gain for screen content videos by introducing two new coding modes, namely, intra block copy (IBC) and palette (PLT) modes. However, the coding gain is achieved at the increased cost of computational complexity. In this paper, we propose a decision tree-based framework for fast intra mode decision by investigating various features in the training sets. To avoid the exhaustive mode searching process, a sequential arrangement of decision trees is proposed to check each mode separately by inserting a classifier before checking a mode. As compared with the previous approaches where both IBC and PLT modes are checked for screen content blocks (SCBs), the proposed coding framework is more flexible which facilitates either the IBC or PLT mode to be checked for SCBs such that computational complexity is further reduced. To enhance the accuracy of decision trees, dynamic features are introduced, which reveal the unique intermediate coding information of a coding unit (CU). Then, if all the modes are decided to be skipped for a CU at the last depth level, at least one possible mode is assigned by a CU-type decision tree. Furthermore, a decision tree constraint technique is developed to reduce the rate-distortion performance loss. Compared with the HEVC-SCC reference software SCM-8.3, the proposed algorithm reduces computational complexity by 47.62% on average with a negligible Bjøntegaard delta bitrate (BDBR) increase of 1.42% under all-intra (AI) configurations, which outperforms all the state-of-the-art algorithms in the literature.

I. Introduction

Screen content video is an emerging video type due to the fast development of the Internet and wireless communication, and it has been applied to many applications, such as online education, remote desktop, and web conferencing [1]. Screen content videos often show a mixed content with both nature image blocks (NIBs) and computer-generated screen content blocks (SCBs) in a single frame, as shown in Fig. 1. Compared with NIBs, SCBs exhibit different characteristics, including no sensor noise, large flat areas with a single color, repeated patterns and limited colors. While NIBs can be well compressed by the conventional intra (Intra) mode in High Efficiency Video Coding (HEVC) [2], new techniques are necessary for SCBs. Therefore, the Joint Collaborative Team on Video Coding (JCT-VC) has developed Screen Content Coding (SCC) extension [3] on top of HEVC to explore new encoding tools for screen content videos since January 2014, and it was finalized in 2016. Fig. 1.

NIB and SCB in the first frame of “WebBrowsing”.