Symmetric Parallax Attention for Stereo Image Super-Resolution

Although recent years have witnessed the great advances in stereo image super-resolution (SR), the beneficial information provided by binocular systems has not been fully used. Since stereo images are highly symmetric under epipolar constraint, in this paper, we improve the performance of stereo image SR by exploiting symmetry cues in stereo image pairs. Specifically, we propose a symmetric bi-directional parallax attention module (biPAM) and an inline occlusion handling scheme to effectively interact cross-view information. Then, we design a Siamese network equipped with a biPAM to super-resolve both sides of views in a highly symmetric manner. Finally, we design several illuminance-robust losses to enhance stereo consistency. Experiments on four public datasets demonstrate the superior performance of our method. Source code is available at https://github.com/YingqianWang/iPASSR.

1. Introduction

With recent advances in stereo vision, dual cameras are commonly adopted in mobile phones and autonomous vehicles. Using the complementary information (i.e., cross-view information) provided by binocular systems, the resolution of image pairs can be enhanced. However, it is challenging to achieve good performance in stereo image super-resolution (SR) due to the following issues: 1) Varying parallax. Objects at different depths have different disparity values and thus locate at different positions along the horizontal epipolar line. It is challenging to capture reliable stereo correspondence and effectively integrate cross-view information for stereo image SR. 2) Information incorporation. Since context information within a single view (i.e., intra-view information) is crucial and contributes to stereo image SR in a different manner, it is important but challenging to fully incorporate both intra-view and cross-view information. 3) Occlusions & boundaries. In occlusion and boundary areas, pixels in one view cannot find their correspondence in the other view. In this case, only intra-view information is available for stereo image SR. It is challenging to fully use cross-view information in non-occluded regions while maintaining promising performance in occluded regions.