Prenc — Predict Number of Video Encoding Passes with Machine Learning

Video streaming providers spend huge amounts of processing time to get a quality-optimized encoding. While the quality-related impact may be known to the service provider, the impact on video quality is hard to assess, when no reference is available. Here, bitstream-based video quality models may be applicable, delivering estimates that include encoding-specific settings. Such models typically use several input parameters, e.g. bitrate, framerate, resolution, video codec, QP values and more. However, for a given bitstream, to determine which encoding parameters were selected, e.g., the number of encoding passes, is not a trivial task. This leads to our following research question: Given an unknown video bitstream, which encoding settings have been used? To tackle this reverse engineering problem, we introduce a system called prenc. Besides the use in video-quality estimation, such algorithms may also be used in other applications such as video forensics. We prove our concept by applying prenc to distinguish between one- and two-pass encoding. Starting from modeling the problem as a classification task, estimating bitstream-based features, we further describe a machine learning approach with feature selection to automatically predict the number of encoding passes for a given video bitstream. Our large-scale evaluation consists of 16 short movie type 4K videos that were segmented and encoded with different settings (resolutions, codecs, bitrates), so that we in total analyzed 131.976 DASH video segments. We further show that our system is robust, based on a 50% train and 50% validation approach without source video overlapping, where we get a classification performance of 65% F1 score. Moreover, we also describe the used bitstream-based features in detail, the feature pooling strategy and include other machine learning algorithms in our evaluation.