Efficient migration of complex off-line computer vision software to real-time system implementation on generic computer hardware

This paper addresses the problem of migrating large and complex computer vision code bases that have been developed off-line, into efficient real-time implementations avoiding the need for rewriting the software, and the associated costs. Creative linking strategies based on Linux loadable kernel modules are presented to create a simultaneous realization of real-time and off-line frame rate computer vision systems from a single code base. In this approach, systemic predictability is achieved by inserting time-critical components of a user-level executable directly into the kernel as a virtual device driver. This effectively emulates a single process space model that is nonpreemptable, nonpageable, and that has direct access to a powerful set of system-level services. This overall approach is shown to provide the basis for building a predictable frame-rate vision system using commercial off-the-shelf hardware and a standard uniprocessor Linux operating system. Experiments on a frame-rate vision system designed for computer-assisted laser retinal surgery show that this method reduces the variance of observed per-frame central processing unit cycle counts by two orders of magnitude. The conclusion is that when predictable application algorithms are used, it is possible to efficiently migrate to a predictable frame-rate computer vision system.

I. Introduction

Computer VISION algorithms have rapidly matured over the past decade, both in terms of sophistication and the range of realistic applications. We are particularly interested in algorithms for real-time frame-rate processing/analysis of image sequences (e.g., digital video) for use in guided surgical instrumentation. In these systems, a digital video camera is used to capture images of a surgical scene, at frame rates ranging from 15 to 200/s. These image sequences are analyzed in real-time to extract quantitative information that can be used to monitor the surgical procedure, perform spatial dosimetry, track structures, compensate for motion, detect hazards, and generate control signals for surgical tool guidance.