Multi-scale and Bi-path Method Based on Image Entropy and CNN for Fast CU Partition in VVC

The latest video compression standard Versatile Video Coding standard (H.266/VVC) has been released by Joint Video Exploration Team. It reaches higher encoding efficiency than the previous standard H.265/HEVC, but it takes more computational resources because it introduces lots of complex tools. Especially the new partition structure named quadtree with nested multi-type tree (QTMT). In that case, a multi-scale bi-path method based on image entropy and CNN for fast partition is proposed in this paper. Considering that CU with different sizes contain different amounts of information, the proposed method sets two paths for CU with large size (32x32) and CU with small size (16x8,8x8,16x16) respectively. The homogeneous area tends to use a larger CU as a whole to encode. So, for large CU, the method uses image entropy and gradient feature to make a partition decision, In HEVC, the CU can only be quartered by Quaternary tree (QT) with a same width and height of 64,32,16 or 8. Besides, large CU contains more specific spatial information, so more concrete partition result is given by a two-step binary classification framework, the method first decides if the CU uses horizontal partition mode or vertical partition mode by gradient distribution, then decide if current CU uses binary partition mode or not by sub-area similarity. Compared to anchor VTM-13.0, the method saves the encoding time by 54.647% on average at the cost of only a 2.27% BDBR increase.