Dual Bidirectional Feature Enhancement Network for Continuous Space-Time Video Super-Resolution

Space-time video super-resolution aims to reconstruct the high-frame-rate and high-resolution video from the corresponding low-frame-rate and low-resolution counterpart. Currently, the task faces the challenge of efficiently extracting long-range temporal information from available frames. Meanwhile, existing methods can only produce results for a specific moment and cannot interpolate high-resolution frames for consecutive time stamps. To address these issues, we propose a multi-stage feature enhancement method that better utilizes the limited spatio-temporal information subject to the efficiency constraint. Our approach involves a pre-alignment module that extracts coarse aligned features from the adjacent odd-numbered frames in the first stage. In the second stage, we use a bidirectional recurrent module to refine the aligned features by exploiting the long-range information from all input frames while simultaneously performing video frame interpolation. The proposed video frame interpolation module concatenates temporal information with spatial features to achieve continuous interpolation, which refines the interpolated feature progressively and enhances the spatial information by utilizing the features of different scales. Extensive experiments on various benchmarks demonstrate that the proposed method outperforms state-of-the-art in both quantitative metrics and visual effects.

I. Introduction

With the development of convolution neural networks (CNNs), the task of space-time video super-resolution (STVSR) has achieved considerable performance. Different from the video super-resolution (VSR), the goal of STVSR is to reconstruct high-frame-rate (HFR) and high-resolution (HR) videos from corresponding low-frame-rate (LFR) and low-resolution (LR) counterparts. STVSR has the potential to be used in a wide range of practical applications and thus has gained increasing interest among academia and industry.