Learning for Unconstrained Space-Time Video Super-Resolution

Recent years have seen considerable research activities devoted to video enhancement that simultaneously increases temporal frame rate and spatial resolution. However, the existing methods either fail to explore the intrinsic relationship between temporal and spatial information or lack flexibility in the choice of final temporal/spatial resolution. In this work, we propose an unconstrained space-time video super-resolution network, which can effectively exploit space-time correlation to boost performance. Moreover, it has complete freedom in adjusting the temporal frame rate and spatial resolution through the use of the optical flow technique and a generalized pixelshuffle operation. Our extensive experiments demonstrate that the proposed method not only outperforms the state-of-the-art, but also requires far fewer parameters and less running time.

I. Introduction

Recently, we have witnessed the popularization of Ultra-High-Definition TeleVision (UHDTV) and the rising of UHD TV shows in broadcasting. However, despite new media contents can be filmed by the advanced UHD recorder, remaking a large quantity of existed ones is impractical, leading to the overall short supply. Video Super-Resolution (VSR) technologies provide a promising way to reconstruct High-Resolution (HR) videos from their Low-Resolution (LR) counterparts. Furthermore, while watching sport events on TV, one may playback the fleeting moments with slow motion. Video Frame Interpolation (VFI) is one of the solutions that can temporally increase the frame rate of the broadcast videos.