Block Matching using ResNet Features

Block matching is one of the popular methods for motion estimation problems. This block-based technique divides an image into macroblocks and finds the movement of each macroblock within a specified search window. This technique uses the pixel values of an image to obtain the motion vectors. In this paper, features extracted using a Convolutional Neural Network (CNN) feature extractor are used instead of the direct pixel values. Compared to pixels, features extracted with a CNN are less affected by small changes in lighting and position within an image. Modern CNNs, such as the ResNet, capture different sizes of features, some finer and some coarser. Compared to directly using pixel values, this method improves the accuracy of matched blocks while reducing the complexity of the block matching algorithm.