Variational AutoEncoder for Reference based Image Super-Resolution

In this paper, we propose a novel reference based image super-resolution approach via Variational AutoEncoder (RefVAE). Existing state-of-the-art methods mainly focus on single image super-resolution which cannot perform well on large upsampling factors, e.g., 8×. We propose a reference based image super-resolution, for which any arbitrary image can act as a reference for super-resolution. Even using random map or low-resolution image itself, the proposed RefVAE can transfer the knowledge from the reference to the super-resolved images. Depending upon different references, the proposed method can generate different versions of super-resolved images from a hidden super- resolution space. Besides using different datasets for some standard evaluations with PSNR and SSIM, we also took part in the NTIRE2021 SR Space challenge [29] and have provided results of the randomness evaluation of our approach. Compared to other state-of-the-art methods, our approach achieves higher diverse scores.

1. Introduction

Image Super-Resolution (SR) is a fundamental problem in image processing. Given a low-resolution (LR) image, the objective is to upsample the LR image by α× to obtain the SR image. In other words, each LR pixel is used for predicting α ×α HR pixels. In real applications, super-resolution is useful for images/videos display, storage, broadcasting and transmission. In particular, it can provide better visual experience for watching movies and playing video games. [2], [3]