A Low-Cost Hardware Design of Learning-Based One-Dimensional Interpolation for Real-Time Video Applications at the Edge

Gradual advances have occurred in the design of micro-artificial intelligence for resource-limited hardware. A high-resolution (HR) image reconstruction module is indispensable for video analytics chips or devices at the edge. This paper proposes a low-cost and learning-based interpolation method for HR image reconstruction. The proposed method generates reconstructed pixels by processing reference pixels with optimal weights, which are pre-trained by solving the minimum mean square error problem for real images. To reduce the number of computation units and the usage of learned weights, a cross-directional interpolation architecture, which includes a vertical kernel and a horizontal kernel, is adopted. Moreover, a one-dimensional feature discriminator is proposed to improve the quality of up-scaled images efficiently. The main benefit of the proposed method is that it requires a small number of computation units but can still produce high-quality images. The hardware architecture of the proposed method was implemented on a field-programmable gate array (FPGA) by using Xilinx UltraScale+ ZCU102 and an application-specific integrated circuit (ASIC) by using TSMC’s 0.13- $\mu \text{m}$ technology. On the ASIC, the proposed hardware required only approximately 60K gate counts and 50 KBytes of memory. The experimental results indicate that the average peak signal-to-noise ratio of the up-scaled images reached 35.92 dB for the Set-5 dataset. The throughput of the proposed hardware was at least 1000 Mpixels/s on the FPGA and 1200 Mpixels/s on the ASIC, which indicates that the proposed hardware can handle a target resolution higher than 4K in real time.

I. Introduction

Artificial intelligence (AI) applications have begun to be gradually deployed in embedded systems; however, many challenges remain in the implementation of large AI models on resource-limited hardware. Therefore, the micro-AI of a chip or device has become intensively investigated.