Mask-VGG: A Shot Scale Classification Model Based on Mask Generation

The development of artificial intelligence has made it possible to automate video production. Shot scale is an important visual characteristic, and intelligent shot scale classification is the key to automate video production. The convolutional neural network has a good performance in the field of image classification. However, when it comes to shot scale classification, a limitation arises in terms of the model's inability to pay attention to the subject of the frame. In the existing schemes, a foreground segmentation model is used to separate the subject from the background, and then shot scale classification is done by a classification model. This “two-step” scheme is computationally expensive and inefficient. In this paper, a Mask-VGG is proposed to classify shot scales in one step, in which a mask generated from the feature map could effectively strengthen the foreground information. And the convolutional layer is to retain the position features of the subject in the shot scale classification task. The proposed model improves the performance of shot scale classification, and the accuracy reaches 95.43%. It also matches the requirement of the live video production in a speed of 6 fps.