Event-and Frame-Based Visual-Inertial Odometry With Adaptive Filtering Based on 8-DOF Warping Uncertainty

In this letter, we present an event- and frame-based visual-inertial odometry (VIO) algorithm that fuses frames, events, and inertial measurement in a robust and adaptive manner. Frames from standard cameras provide rich context of the scene at a fixed rate. Event cameras on the other hand asynchronously produce events at a pixel-level when changes in intensity occur, and thus are resilient to motion blur and have high dynamic range. To harness the advantages of the two sensors, our frontend fuses their outputs by creating brightness increment patches of each output and minimize the differences with an 8-DOF warping model. The warping model and the optimization process allow for robust feature tracking in the frontend of the algorithm. The minimized residual is then used in the multi-state filter-based backend where the measurement update is adaptively performed depending on the size of the residual for accurate estimation, reflecting the quality of the tracked features. Comparative evaluation on two publicly available datasets reveals that our method outperforms the state-of-the-art event-based VIO algorithms in pose estimation accuracy.

I. Introduction

Accurately estimating the ego-motion of a sensor is a crucial factor in various applications, including autonomous robotics and augmented/virtual reality. Recently, numerous vision-based algorithms have been proposed to solve such problem and have shown great progress. Yet, conventional cameras possess innate limitations of motion blur and low dynamic range. In situations of high-speed motions or environments with high dynamic range, the navigation algorithms using only the standard cameras perform underwhelmingly or even fail.