Infrared and Visible Image Registration for Airborne Camera Systems

Current airborne camera systems (ACS) used for surveillance and reconnaissance are generally equipped with cameras operating in different wavelengths. The registration and fusion of the images generated by these cameras enable a multimodal detection capability. In this paper, we propose a method to perform the geometric registration of infrared and visible imagery collected with an electro-optical and infrared (EO-IR) ACS. The registration process is a two-step approach. First, an initial transformation is computed using the expected orientation of the scene with respect to the camera and the camera’s intrinsic and extrinsic parameters. Then, the image alignment is refined using a technique based on mutual information which adapts the registration to the scene’s geometry and copes with video streams that are not perfectly synchronized. The image registration technique is implemented on an edge computing device and currently runs at 8 Hz. The intent is to use the resulting system to augment the capability of ACS that do no have built-in image registration and fusion capabilities. We present results obtained by applying the registration approach on real airborne imagery. We also evaluate the image registration error using physical markers installed on a building.