COUPLE: Orchestrating Video Analytics on Heterogeneous Mobile Processors

Video analytics is considered the killer application of edge computing and has been successfully deployed across diverse domains. Yet, executing video analytics on mobile devices presents notable challenges owing to the considerable computational demands and frame rate requirements of DNN models. Current mobile inference frameworks often concentrate on enhancing model inference performance on the CPU or GPU, overlooking the potential of the Digital Signal Processor (DSP) – an emerging heterogeneous processor increasingly integrated into modern mobile processors. In this paper, we introduce COUPLE, an orchestration framework for video analytics on heterogeneous mobile processors, with the goal of optimizing real-time video analysis through the collaboration of CPU, GPU and DSP. To tackle the accuracy loss of DSP inference, we introduce the Anchor Frame Calibration mechanism, utilizing high-precision GPU inference results and frame similarities to mitigate accuracy loss on the DSP. Additionally, we design a lightweight progressive scheduler to distribute video frames to GPU and DSP, maximizing inference Average Precision (AP) under performance (i.e., frame rate) and power constraints. COUPLE has been implemented on the Qualcomm's Snapdragon 888 mobile SoC, extensive evaluation results demonstrate its efficacy in imnroving the inference performance and accuracy.