GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild

Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving as a partial observation of the High Dynamic Range (HDR) visual world. Despite limited dynamic range, these LDR images are often captured with different exposures, implicitly containing information about the underlying HDR image distribution. Inspired by this intuition, in this work we present, to the best of our knowledge, the first method for learning a generative model of HDR images from in-the-wild LDR image collections in a fully unsupervised manner. The key idea is to train a generative adversarial network (GAN) to generate HDR images which, when projected to LDR under various exposures, are indistinguishable from real LDR images. The projection from HDR to LDR is achieved via a camera model that captures the stochasticity in exposure and camera response function. Experiments show that our method GlowGAN can synthesize photorealistic HDR images in many challenging cases such as landscapes, lightning, or windows, where previous supervised generative models produce overexposed images. With the assistance of GlowGAN, we showcase the novel application of unsupervised inverse tone mapping (GlowGAN-ITM) that sets a new paradigm in this field. Unlike previous methods that gradually complete information from LDR input, GlowGAN-ITM searches the entire HDR image manifold modeled by GlowGAN for the HDR images which can be mapped back to the LDR input. GlowGAN-ITM achieves more realistic reconstruction of overexposed regions compared to state-of-the-art supervised learning models, despite not requiring HDR images or paired multi-exposure images for training.

1. Introduction

High Dynamic Range (HDR) images [57] are capable of capturing and displaying much richer appearance information than Low Dynamic Range (LDR) images, thus playing an important role in image representation and visualization. The most popular method to acquire HDR images is multiple exposure blending, which requires capturing a set of LDR images of the same scene with different exposures [13], [52], [59]. However, this is time and effort intensive and only suitable for static scenes. Due to this limitation, existing HDR image datasets only cover limited scene categories and have much fewer images than LDR datasets. Thus, supervised learning methods [16], [47], [14], [41], [45], [43], [19], [71], [70], [28] that reconstruct an HDR image from an LDR image are constrained by the HDR datasets and cannot extend to cases where no HDR training data is available, e.g., lightnings or campfires.