Fusing Multiple Isolated Maps to Visual Inertial Odometry Online: A Consistent Filter

Visual inertial odometry (VIO) is widely used in various kinds of mobile platforms to provide the ego-pose of the platforms. With the help of pre-built map information, the drift of the VIO can be constrained. However, constructing a globally consistent map is a tough job, especially for large scenes. In this paper, we propose a filter-based framework aiming to leverage multiple isolated maps to improve the performance of VIO such that building a globally consistent map can be avoided. In this framework, the relative transformations between the local VIO reference frame and the multiple map reference frames are regarded as 6 degrees of freedom (DoF) pose features to be online estimated. We call these relative transformations as augmented variables. With these augmented variables, the map-based information can be tightly coupled into the VIO system to ease the drift of VIO. To fuse these maps consistently, we first theoretically analyze the observability properties of our proposed framework. Based on the analysis, the Schmidt extended Kalman filter (EKF) and the first-estimate Jacobian (FEJ) are employed to maintain the consistency of the system. Simulation and real-world experiments are conducted to demonstrate the effectiveness and consistency of our framework. Note to Practitioners—Visual inertial odometry (VIO) is widely applied to positioning mobile platforms including autonomous vehicles, robots, and virtual/augmented reality (VR/AR) devices. However, VIO inevitably suffers from drift, which will reduce positioning accuracy. This problem can be solved by fusing prior maps into VIO. Existing works mainly support online fusing one map into VIO. This requires users to offline merge multiple maps into one beforehand, which is complicated and troublesome and sometimes even unrealizable (e.g., the multiple maps have no overlap). According to theoretical analyses, this paper introduces a new system that can online fuse multiple maps into VIO. Our system has the following benefits: 1) It is a light-weighted filter-based system suitable for onboard deployments; 2) It can online fuse multiple maps such that pre-work of merging multiple maps into one can be bypassed; 3) Our system can consistently fuse the multi-map information while keeps the computation at a low level. Experiments show that with our system, the VIO’s drift can be significantly alleviated to benefit downstream tasks like planning, navigation, and control. However, our system needs to fix some linearization points to maintain the correct observability of the system, which will sacrifice some precision. Future works will include investigating more elegant techniques to maintain the observability of the system.