Hybrid Cross-View Attention Network for Lightweight Stereo Image Super-Resolution

The goal of stereo image super-resolution is to enhance the quality of low-resolution stereo image pairs by utilizing complementary information across views. Although transformer-based methods have shown high efficiency in single-image super-resolution tasks, they have not been fully used in stereo super-resolution tasks. Therefore, it is crucial to incorporate the complementary information of stereo images into the transformer method to improve image details. To address this challenge, we propose a lightweight Hybrid Cross-view Attention Stereo Super-Resolution network (HCASSR), which uses a Transformer-based network for intra-view feature extraction and a cross-view attention module to aggregate stereo image information. We also employ multi-stage training strategies and data ensemble in test-time to improve image quality. Our method has been extensively tested on the KITTI 2012, KITTI 2015, Middlebury, and Flickr1024 datasets, and the experimental results demonstrate that the proposed method outperforms existing works with smaller model size. Additionally, we won 3rd and 2nd place respectively in Track 1 and Track 2 of the NTIRE 2024 Stereo Image Super-Resolution Challenge. Codes and models will be released at https://github.com/YuqiangY/HCASSR.

1. Introduction

Stereo image super-resolution technology is an advanced image processing method whose core goal is to reconstruct a more detailed high-resolution image from a pair of low- resolution stereo views (i.e., left and right views). In many applications like AR/VR and robot navigation, increasing the resolution of stereo images is highly demanded to achieve higher perceptual quality and help to parse the real world. Therefore, this technology has developed rapidly in recent years and has shown great application potential and widespread attention in many fields. Stereo image super-resolution and single image super-resolution have essential similarities, but there is a key difference between them. Single image super-resolution is limited to extracting information from one perspective, while stereo image super-resolution can integrate information from two views with large overlapping areas. This is crucial because information that may be missing from one view may still exist in another view. Therefore, by effectively integrating information from two perspectives, the quality and details of reconstructed images can be significantly improved, which is a key factor in the success of stereo image super-resolution methods.