Learning Parallax Attention for Stereo Image Super-Resolution

Stereo image pairs can be used to improve the performance of super-resolution (SR) since additional information is provided from a second viewpoint. However, it is challenging to incorporate this information for SR since disparities between stereo images vary significantly. In this paper, we propose a parallax-attention stereo superresolution network (PASSRnet) to integrate the information from a stereo image pair for SR. Specifically, we introduce a parallax-attention mechanism with a global receptive field along the epipolar line to handle different stereo images with large disparity variations. We also propose a new and the largest dataset for stereo image SR (namely, Flickr1024). Extensive experiments demonstrate that the parallax-attention mechanism can capture correspondence between stereo images to improve SR performance with a small computational and memory cost. Comparative results show that our PASSRnet achieves the state-of-the-art performance on the Middlebury, KITTI 2012 and KITTI 2015 datasets.

1. Introduction

Super-resolution (SR) aims to reconstruct high-resolution (HR) images from their low-resolution (LR) counterparts. Recovering an HR image from a single shot is a long-standing problem [1], [2], [3]. Recently, dual cameras are becoming increasingly popular in mobile phones and autonomous vehicles. It is already demonstrated that subpixel shifts contained in LR stereo images can be used to improve SR performance [4]. However, since disparities between stereo images can vary significantly for different baselines, focal lengths, depths and resolutions, it is highly challenging to incorporate stereo correspondence for SR.