Image Transformation Based on Learning Dictionaries across Image Spaces

In this paper, we propose a framework of transforming images from a source image space to a target image space, based on learning coupled dictionaries from a training set of paired images. The framework can be used for applications such as image super-resolution and estimation of image intrinsic components (shading and albedo). It is based on a local parametric regression approach, using sparse feature representations over learned coupled dictionaries across the source and target image spaces. After coupled dictionary learning, sparse coefficient vectors of training image patch pairs are partitioned into easily retrievable local clusters. For any test image patch, we can fast index into its closest local cluster and perform a local parametric regression between the learned sparse feature spaces. The obtained sparse representation (together with the learned target space dictionary) provides multiple constraints for each pixel of the target image to be estimated. The final target image is reconstructed based on these constraints. The contributions of our proposed framework are three-fold. 1) We propose a concept of coupled dictionary learning based on coupled sparse coding which requires the sparse coefficient vectors of a pair of corresponding source and target image patches to have the same support, i.e., the same indices of nonzero elements. 2) We devise a space partitioning scheme to divide the high-dimensional but sparse feature space into local clusters. The partitioning facilitates extremely fast retrieval of closest local clusters for query patches. 3) Benefiting from sparse feature-based image transformation, our method is more robust to corrupted input data, and can be considered as a simultaneous image restoration and transformation process. Experiments on intrinsic image estimation and super-resolution demonstrate the effectiveness and efficiency of our proposed method.

1 Introduction

Many problems in computer vision can be cast as transforming images from one form to another. In this work, we assume that the pairs of images have the same content and are in registration with each other, but are visually different. Typical applications include, but are not limited to, super-resolution (SR), face mapping of different styles, estimation of intrinsic images, such as shading and albedo images, and various nonphotorealistic rendering. Some existing methods generate stylistic images based on a single or a pair of reference images [3], [1], [2]. Others learn the mapping relations from large training sets of paired images [4], [6], [7], [8], [49], [51]. Our work belongs to the second category. A few representative methods of such kind are summarized as follows.