Shareable camera framework for multiple computer vision applications

On IoT devices such as autonomous driving drones, computer vision jobs such as video recording, streaming and object detection use same camera frame. However, since these IoT devices are resource-constrained systems, they have two problems. First, these applications often do duplicated processing for the same camera raw frame. Second, scheduling between computer vision applications is difficult. In this paper, we propose a shareable camera framework that performs the tasks of computer vision applications. This framework converts the existing pipeline to a pipeline that does not have redundant processing based on the data flow whenever it receives a request from the applications. It also has a scheduling algorithm to guarantee quality-of-service of the applications in the resource-constrained systems. With the proposed framework, the IoT application developers can easily develop reliable computer vision applications that share a single camera simultaneously.

I. Introduction

On loT devices such as autonomous driving drones, computer vision tasks such as video recording, streaming and object detection use same camera frame. Deep neural networks (DNN) are generally used for computer vision tasks, which consume more significant computing resources such as CPU computation and memory than traditional computer vision tasks. In order to execute DNN for computer vision tasks, high-performance computing systems are required.