Correspondence Between SWIR and MWIR Images Using Augmentation and Preprocessing for Registration

Multispectral sensors are used to ensure visibility in various applications. However, when multiple sensors are used for capturing images, a misalignment may occur between the images taken by each sensor unless special care is taken. To correct such misalignments, image registration based on feature matching is conducted. However, the features captured by each sensor differ, thereby complicating the registration process. In this study, we develop an approach to overcome these challenges and to improve the registration accuracy between short-wave infrared and mid-wave infrared (SWIR and MWIR, respectively) images. First, we compare and validate SiLK, a detector-based feature matching method, and LoFTR, a detector-free feature matching method. The results clearly demonstrate the superior accuracy of LoFTR. Moreover, SWIR and MWIR images exhibit a characteristic color inversion according to Kirchhoff’ s law. Therefore, by inverting the color of a SWIR image and aligning the color tone between image pairs, we can improve the matching accuracy. Furthermore, by diversifying the color tones of the training data through augmentation, we can handle the domain gap between SWIR and MWIR images, thereby further enhancing the matching accuracy.