Speed Up H.266/QTMT Intra-Coding Based on Predictions of ResNet and Random Forest Classifier

The video codec, H.266/VVC, adopts a Quad-Tree-plus-Multi-type-Tree(QTMT). It's efficient but with high time complexity. A convolutional-neural-network(CNN) and a random-forest-classifier are designed to predict the depth and splitting-type of a 32 × 32 Coding-Unit to eliminate exhaustive RDO-tests. The CNN outputs a label that specifies a depth-range for the coder to early-skip or early-terminate coding; We utilize a random-forest(RF) algorithm to design six RF binary classifiers for a multi-level one. If it classifies a CU to be not of its splitting-type, it omits corresponding RDO-tests. Experiments showed, compared with VTM7.0, the proposed method can reduce 39.16% of execution time with 0.7% BDBR increment.