Generating Visible Spectrum Images from Thermal Infrared using Conditional Generative Adversarial Networks

Thermal Infrared cameras have a unique set of capabilities that enables them to see and capture images in situations where vision is severely obscured, including dark, shade, and foggy conditions. Conversion of TIR image to visible spectrum makes them more interpretable, but automating transformation is a challenging task as it requires estimation of both chrominance and luminance for each pixel, posing the need for a more complex model that is capable of capturing higher-level semantics of a TIR image to generate a perceptually realistic RGB image. The use of Conditional Generative Adversarial Networks (cGANs) is proposed with a combination of content and adversarial losses to tackle this problem. The proposed cGAN uses a U-Net auto-encoder network as generator, and a patch based discriminator for adversarial training. The cGAN learns the most effective loss function during training, producing state-of-the-art perceptually realistic outputs. Qualitative and quantitative analyses show that our approach outperforms existing methods.