Fully End-to-End Learning Based Conditional Boundary Equilibrium GAN with Receptive Field Sizes Enlarged for Single Ultra-High Resolution Image Dehazing

A receptive field is defined as the region in an input image space that an output image pixel is looking at. Thus, the receptive field size influences the learning of deep convolution neural networks. Especially, in single image dehazing problems, larger receptive fields often show more effective dehazying by considering the brightness and color of the entire input hazy image without additional information (e.g. scene transmission map, depth map, and atmospheric light). The conventional generative adversarial network (GAN) with small-sized receptive fields cannot be effective for hazy images of ultra-high resolution. Thus, we proposed a fully end-to-end learning based conditional boundary equilibrium generative adversarial network (BEGAN) with the receptive field sizes enlarged for single image dehazing. In our conditional BEGAN, its discriminator is trained ultra-high resolution conditioned on downscale input hazy images, so that the haze can effectively be removed with the original structures of images stably preserved. From this, we can obtain the high PSNR performance (Track 1 - Indoor: top 4th-ranked) and fast computation speeds. Also, we combine an L1 loss, a perceptual loss and a GAN loss as the generator's loss of the proposed conditional BEGAN, which allows to obtain stable dehazing results for various hazy images.

1. Introduction

Images are often captured under bad weather conditions, which results in the degraded images with many obscured regions by fog, mist, and haze etc. Especially, the hazy images not only lower their aesthetical values, but also cause a significant performance degradation for object recognition. Thus, dehazing is an essential preprocessing to both aesthetic photography and computer vision applications. In general, the formulation of a hazy image can be modeled as

$$\begin{equation*} I(x)=J(x)t(x)+A(1-t(x)) \tag{1} \end{equation*}$$ where and are an input hazy image and a clean image, is the global atmospheric light, and is the transmission ratio that the potion of lights reaches the camera sensors. As a result, the haze removal using only a single degraded hazy image is a very challenging and ill-posed problem. The conventional haze removal methods estimate the global atmospheric light and the transmission ratio, and they remove the haze using the estimated parameters of (1) [1]–[4]. But, this approach is not a way to optimize the perceptual quality of generated dehazed images. Also, the inaccuracies of the estimated parameters can lead to weird distortions or to poor performance of haze removal. Instead, deep-learning-based convolutional neural networks can be used to effectively remove the image haze via fully end-to-end-learning. For an effective fully-end-to-end learning, the network must be able to understand the characteristics of the entire hazy images. Especially, when the resolutions of hazy images are very large, the training of the haze removal networks with small receptive filed sizes becomes difficult since the networks cannot consider the properties of the entire hazy images.