Look-Up Table Compression for Efficient Image Restoration

Look-Up Table (LUT) has recently gained increasing at-tention for restoring High-Quality (HQ) images from Low-Quality (LQ) observations, thanks to its high computational efficiency achieved through a “space for time” strategy of caching learned LQ-HQ pairs. However, incorporating multiple LUTs for improved performance comes at the cost of a rapidly growing storage size, which is ultimately re-stricted by the allocatable on-device cache size. In this work, we propose a novel LUT compression framework to achieve a better trade-off between storage size and performance for LUT-based image restoration models. Based on the observation that most cached LQ image patches are dis-tributed along the diagonal of a LUT, we devise a Diagonal-First Compression (DFC) framework, where diagonal LQ-HQ pairs are preserved and carefully re-indexed to main-tain the representation capacity, while non-diagonal pairs are aggressively subsampled to save storage. Extensive ex-periments on representative image restoration tasks demon-strate that our DFC framework significantly reduces the storage size of LUT-based models (including our new de-sign) while maintaining their performance. For instance, DFC saves up to 90% of storage at a negligible performance drop for x 4 super-resolution. The source code is available on GitHub: https://github.com/leenas233IDFC.

1. Introduction

Image restoration, such as super-resolution, denoising, de-blocking, and deblurring, aims at reconstructing high-quality (HQ) images with rich high-frequency details from low-quality (LQ) degraded observations. In recent years, with the rapid development of deep learning, methods based on deep neural networks (DNN) [2], [13], [19], [22], [29], [50], [51] have made impressive progress in image restoration. Nev-ertheless, these methods often require high computational costs or dedicated computing devices, e.g., GPUs and TPUs. Such an excessive computational requirement limits the usage of DNN-based methods on edge devices with limited resources, such as smartphones and televisions. Figure 1.

Performance-storage trade-off for x 4 super-resolution on the Set5 dataset [3]. Our proposed DFC effectively compresses the storage size of the LUT-based models with a high compression ratio while maintaining performance. The orange dotted line and blue dotted line indicate the L2 cache and L3 cache sizes of a Qualcomm Snapdragon 888 Plus chip respectively.