Single Image Super-Resolution Using Depth Map as Constraint

Single image super-resolution based on the deep neural network has achieved great performance recently, but generating photo-realistic images remains a challenging problem. To tackle this issue, we propose a method that uses depth maps as a constraint to get better visual quality. Specifically, we propose a self-adaptive feature transform (AFT) layer, which can perform affine transformation on the feature map based on the depth map to constrain the plausible solution space of the SR image. Furthermore, we propose a hierarchical residual multi-scale fusion block to improve the representational ability of the network. Experimental results on benchmark datasets demonstrate that our method is superior to other perceptually-oriented SISR methods in terms of visual quality and also achieves state-of-the-art performance on quantitative metrics.