TP-TIO: A Robust Thermal-Inertial Odometry with Deep ThermalPoint

To achieve robust motion estimation in visually degraded environments, thermal odometry has been an attraction in the robotics community. However, most thermal odometry methods are purely based on classical feature extractors, which is difficult to establish robust correspondences in successive frames due to sudden photometric changes and large thermal noise. To solve this problem, we propose ThermalPoint, a lightweight feature detection network specifically tailored for producing keypoints on thermal images, providing notable anti-noise improvements compared with other state-of-the-art methods. After that, we combine ThermalPoint with a novel radiometric feature tracking method, which directly makes use of full radiometric data and establishes reliable correspondences between sequential frames. Finally, taking advantage of an optimization-based visual-inertial framework, a deep feature-based thermal-inertial odometry (TP-TIO) framework is proposed and evaluated thoroughly in various visually degraded environments. Experiments show that our method outperforms state-of-the-art visual and laser odometry methods in smoke-filled environments and achieves competitive accuracy in normal environments.

I. Introduction

Robust and accurate state estimation for micro aerial vehicles (MAVs) is of crucial importance as these versatile robots are taking the place of people to fulfill complex and dangerous missions, such as industrial inspection, remote sensing, search and rescue. Due to their flexibility and ability to minimize risks to humans, there is an increased demand for their ability to achieve perception and localization safely and reliably in more challenging environments, such as fire scenes, subterranean settings, as well as GPS-denied environments. To localize in such environments, visible-light camera sensors seem to be a suitable choice for MAVs because of low power consumption, light weight and affordable price. A number of promising visual odometry and visual-inertial odometry methods have been proposed in recent years. Nevertheless, due to the poorly illuminated conditions and airborne obscurants conditions such as dust, fog and smoke in real world, the data of visible-light cameras can significantly degrade, which makes them unreliable for motion estimation.