Super-Interpolation With Edge-Orientation-Based Mapping Kernels for Low Complex $2\times $ Upscaling

With the advent of ultrahigh-definition (UHD) video services, super-resolution (SR) techniques are often required to generate high-resolution (HR) images from low-resolution (LR) images, such as HD images. To generate such HR images and a video of UHD resolutions in limited computing devices with hardware and software, low complex but excellent SR methods are particularly required. In this paper, we present a novel and fast SR method, called super-interpolation (SI), by unifying an interpolation step and a quality-enhancement step. The proposed SI method utilizes edge-orientation (EO)-based pre-learned kernels, which inherits the simplicity of interpolation and the quality enhancement of SR. It performs SR directly from the initial resolution of an input image to the target resolution of an up-scaled output image without requiring any intermediate interpolated image. The proposed SI method involves offline training and online up-scaling phases. In the offline training phase, training LR image patches are clustered based on their edge orientations into different EO classes for which class-dependent linear mapping functions are learned between training LR and HR image patches. In up-scaling phase, an HR output image patch for each LR input image patch is generated by applying an appropriate linear mapping function selected based on the EO of LR input image patch. Our proposed SI method is intensively compared with the ten state-of-the-art SR methods for common image sets and many HD/UHD images. The experimental results show that the SI method yields the smallest running time and requires relatively small hardware resources. It outperforms the six state-of-the-art methods in average (peak signal-to-noise ratio) PSNR/(structural similarity) SSIM, and exhibits competitive or somewhat lower PSNR/SSIM performance compared with the others.

I. Introduction

Uhdtvs (Ultra High Definition Televisions) are now slowly emerging in the consumer markets, thereby growing the needs for UHD video contents. Most of smartphones in near future will have a capability to support UHD video. However, due to costly acquisition devices and large bandwidth occupancy, there are not enough UHD contents available to suffice the UHD services. Also, many legacy contents of low spatial-resolution video are under services via IPTV and Internet. This has attracted the use of up-scaling techniques, super-resolution, which are able to convert small-resolution video to large-resolution ones, especially for video up-scaling by a magnification factor of 2 as in Full High Definition (FHD)-to-UHD or 4K-to-8K conversions [1], [2].