CNN Based CU Partition Mode Decision Algorithm for HEVC Inter Coding

As compared with the predecessors, the superior compression performance of HEVC mainly stems from the hierarchical quadtree coding scheme, which is composed of coding unit(CU), prediction unit(PU), and transform unit(TU). The best CU/PU/TU partition mode is chosen from plenty of candidate modes. This procedure is denoted as rate-distortion optimization(RDO) that consumed more than 90% computation resources in HEVC encoding. In this paper, we devise the convolutional neural network(CNN) based fast CU mode decision algorithm for HEVC inter prediction. The contributions of our proposals include: (1) Because the maximum number of CU/PU candidate mode in one CTU is reduced, the corresponding VLSI encoder hardware complexity is ameliorated; (2) With the CTU pipeline architecture, the parallelism of the RDO processing will not be deteriorated by our fast algorithm. Our experiments show that the proposed VLSI friendly algorithm speeds up the HEVC inter coding by 45.0% at the cost of averagely 2.91 % Bjontegaard Delta bit-rate(BDBR) increase in HEVC reference test model HM-15.0.