Cross-view Aggregation Network For Stereo Image Super-Resolution

Although stereo image super-resolution has been extensively studied, many existing works only rely on attention in a single epipolar direction to reconstruct stereo images. In the case of asymmetric parallax images, these methods often struggle to capture reliable stereo correspondence, resulting in reconstructed images suffering from blurring and artifacts. In this paper, we propose a novel method called Cross-View Aggregation Network for Stereo Image Super-Resolution (CANSSR) and explore the relationship between multi-directional epipolar lines to construct reliable stereo correspondence. Specifically, we propose a multidirectional cross-view aggregation module (MCAM) that effectively captures multi-directional stereo correspondence and obtains cross-view complementary information. Furthermore, we design a channel-spatial aggregation module (CSAM) that aggregates multi-order global-local information in intra-view to reconstruct clearer texture features. In addition, we equip a large kernel convolution in the Feedforward Network to acquire richer detailed texture information. The extensive experiments conclusively demonstrate that CANSSR outperforms the state-of-the-art method both qualitatively and quantitatively in terms of stereo image super-resolution on the Flickr 1024 and Middlebury datasets.

1. Introduction

Recently, there has been a noticeable surge in the utilization of stereo imaging devices, particularly within the domains of dual-lens smartphones, unmanned systems, augmented reality, virtual reality, autonomous driving, and robotics, etc. Stereoscopic vision has received substantial attention from both academia and industry. However, due to the physical imaging limitations [14] of binocular cameras, low-resolution (LR) stereo images pose significant challenges for practical applications [31]. Therefore, reconstructing high-resolution (HR) images is extremely urgent for the stereo vision task. Compared with single image superresolution (SISR), stereo image super-resolution (SR) needs to utilize complementary information in cross-views, and lost or occluded details are restored by leveraging complementary information from the another view image. In practice, due to the binocular camera imaging settings, stereo images often exhibit a horizontal or vertical pixel-level offset, known as horizontal parallax and vertical parallax. Several studies [7] [31] have demonstrated that the parallax effect between LR images induces sub-pixel displacement, which contains huge spatial dependence information in the stereo vision system. However, these methods only utilize horizontal parallax prior and fail to consider vertical parallax prior in order to improve network performance. Therefore, it is crucial to effectively utilize the multi-directional parallax prior for stereo image SR.