Enabling Real-Time Video Analytics with Adaptive Sampling and Detection-Based Tracking in Edge Computing

With the popularization of visual machine learning, intelligent video analytics can automatically analyze and extract information from video streams, yet it brings heavy computing burdens. Edge computing can improve the processing experience by bringing computing resources near users. On top of this, various processing methods and settings have different resource requirements and output different user experiences. How to dynamically select the video processing configuration according to system states becomes a critical problem that remains to be addressed. In this paper, we propose an edge-assisted video analytic framework based on adaptive sampling and detection-based tracking. We design four functional modules to realize a cooperative computing processing flow. We consider two performance metrics, recognition accuracy and processing time to estimate the experience of real-time video analytics. Further, we design an online configuration method based on Double Deep Q-Network, which can adaptively select analytic configurations under the condition of system dynamics. Experimental results based on a real dataset demonstrate the superior performance of the proposed framework on reward, mean Intersection over Union (IoU), and processing time.

I. Introduction

In recent years, we have witnessed the popularization of visual machine learning in the Internet of Things (IoT), which drives a wide range of intelligent video analytic applications, such as traffic control, environment monitoring, and autonomous driving. Critical in these applications is detecting objects from continuously captured video frames. An ideal object detection engine should be accurate and real-time for these deep vision applications. However, most object detection models are based on deep neural networks (DNNs) and place a computational burden on portable devices, which leads to task timeout, energy exhaustion, and other serious issues [1].