High Dynamic Range Imaging for Dynamic Scenes With Large-Scale Motions and Severe Saturation

Two key challenges exist in high dynamic range (HDR) imaging from multiexposure low dynamic range (LDR) images for dynamic scenes: 1) aligning the input images with large-scale foreground motions and 2) recovering large saturated regions from a limited number of input LDR images. To tackle these challenges, several deep convolutional neural networks have been proposed that have made significant progress. However, these methods tend to suffer from ghosting and saturation artifacts when applied to some challenging scenes. In this article, we propose an end-to-end deformable HDR imaging network, called DHDRNet, which attempts to alleviate these problems by building an effective aligning module and adopting self-guided attention. First, we analyze the alignment process in the HDR imaging task and correspondingly design a pyramidal deformable module (PDM) that aligns LDR images at multiple scales and reconstructs aligned features in a coarse-to-fine manner. In this way, the proposed DHDRNet can handle large-scale complex motions and suppress ghosting artifacts caused by misalignments. Moreover, we adopt self-guided attention to reduce the influence of saturated regions during the aligning and merging processes, which helps suppress artifacts and retain fine details in the final HDR image. Extensive qualitative and quantitative comparisons demonstrate that the proposed model outperforms the existing start-of-the-art methods and that it is robust to challenging scenes with large-scale motions and severe saturation. The source code is available at: https://github.com/Tx000/DHDRNet.

I. Introduction

The average luminance level in natural scenes can range from (night) to (daylight) [1]. However, due to hardware limitations, the dynamic range of images captured by standard digital cameras is far lower than the dynamic range of luminance in natural scenes. As a result, high or low luminance areas in the scene will appear as completely white (overexposure) or black (underexposure) areas in captured low dynamic range (LDR) images, which cause information loss and thus greatly decrease the accuracy of image-based measurement methods. In contrast, the high dynamic range (HDR) image has a wider brightness range and significantly improves the visual quality, containing more useful information. In addition, as a measure of relative luminance of the scene, HDR imaging is useful for illustrating some properties of the scene such as the presence of diffuse or specular surfaces and lighting condition [1]. For these reasons, HDR technology has an extensive influence on various applications and related fields [2]–[5].