HDR-GAN: HDR Image Reconstruction From Multi-Exposed LDR Images With Large Motions

Synthesizing high dynamic range (HDR) images from multiple low-dynamic range (LDR) exposures in dynamic scenes is challenging. There are two major problems caused by the large motions of foreground objects. One is the severe misalignment among the LDR images. The other is the missing content due to the over-/under-saturated regions caused by the moving objects, which may not be easily compensated for by the multiple LDR exposures. Thus, it requires the HDR generation model to be able to properly fuse the LDR images and restore the missing details without introducing artifacts. To address these two problems, we propose in this paper a novel GAN-based model, HDR-GAN, for synthesizing HDR images from multi-exposed LDR images. To our best knowledge, this work is the first GAN-based approach for fusing multi-exposed LDR images for HDR reconstruction. By incorporating adversarial learning, our method is able to produce faithful information in the regions with missing content. In addition, we also propose a novel generator network, with a reference-based residual merging block for aligning large object motions in the feature domain, and a deep HDR supervision scheme for eliminating artifacts of the reconstructed HDR images. Experimental results demonstrate that our model achieves state-of-the-art reconstruction performance over the prior HDR methods on diverse scenes.

I. Introduction

Although the human visual system has a much higher visual dynamic range, most off-the-shelf digital cameras typically produce photos with a limited range of illumination, which may not be desirable in many scenarios. Some specialized hardware devices [1], [2] have been proposed to produce high dynamic range (HDR) images directly, but they are usually too expensive to be widely used. In recent years, with prevailing mobile devices, there is a high demand for capturing HDR images of scenes using light-weighted and low-cost monocular mobile cameras, in order to produce photos that cover a broad range of illumination.