AdaDSR: Adaptive Configuration Optimization for Neural Enhanced Video Analytics Streaming

Neural-based super-resolution (SR) has achieved great success in enhancing image or video quality, creating new opportunities for building bandwidth-efficient and high-accuracy video analytics (VAs) systems. Intuitively, with the help of SR techniques, cameras only need to send downsampled low-quality frames to the server in a canonical edge-assisted VAs framework. The server-side SR model then upscales the quality of received frames for the subsequent VAs tasks, incurring thus substantially reduced bandwidth consumption. Nonetheless, as revealed by our measurement results on real-world video clips, higher delivery quality does not necessarily lead to higher analysis accuracy. This motivates us to study the content-adaptive downsampling and upscaling ratio selection problem for VAs streaming. We propose an SR-based VAs framework, named AdaDSR that can dynamically select the optimal downsampling and upscaling ratios so that the system utility can be maximized. AdaSDR is configured to balance the tradeoffs among accuracy, network cost, and computational cost. It further leverages the temporal consistency of videos to skip trivial decisions so that the camera’s processing overhead can be reduced. Experiments on real-world video data sets demonstrate that AdaDSR can improve the average utility by 7.2%–18.4% when compared with state-of-the-art approaches under diverse video scenes.

I. Introduction

With the development of deep neural network (DNN)-based computer vision (CV) models and an increasing number of camera deployments, recent years have witnessed an explosive growth of video analytics (VAs) tasks, such as object detection and semantic segmentation [1], [2], [3]. Currently, most front-end devices stream their captured videos to resource-rich cloud servers to achieve high-analysis accuracy. However, cloud servers typically reside far away from these front-end devices, which introduces significant network consumption for large-volume video transmission. Streaming filtered frames to the cloud server can affect the inference accuracy due to the loss of image details. On the other hand, analyzing videos directly on front-end devices could avoid the network cost, however, it cannot support high-accuracy VAs at need due to the limited computational resources of most front-end devices. Therefore, it is urgent to find a solution to achieve high-accurate VAs with an affordable network cost.