Image up-sampling using total-variation regularization with a new observation model

This paper presents a new formulation of the regularized image up-sampling problem that incorporates models of the image acquisition and display processes. We give a new analytic perspective that justifies the use of total-variation regularization from a signal processing perspective, based on an analysis that specifies the requirements of edge-directed filtering. This approach leads to a new data fidelity term that has been coupled with a total-variation regularizer to yield our objective function. This objective function is minimized using a level-sets motion that is based on the level-set method, with two types of motion that interact simultaneously. A new choice of these motions leads to a stable solution scheme that has a unique minimum. One aspect of the human visual system, perceptual uniformity, is treated in accordance with the linear nature of the data fidelity term. The method was implemented and has been verified to provide improved results, yielding crisp edges without introducing ringing or other artifacts.

I. Introduction

DIGITAL-image magnification with higher perceived resolution is of great interest for many applications, such as law enforcement and surveillance, standards conversions for broadcasting, printing, aerial- and satellite-image zooming, and texture mapping in computer graphics. In such applications, a continuous real-world scene is projected by an ideal (pin-hole) optical system onto an image plane and cropped to a rectangle . The resulting continuous image is acquired by a physical camera to produce a digital lower resolution (LR) image (i.e., lower than desired) defined on a lattice (following the notation of [1], [2]). This camera, including the actual optical component, is modeled as shown in Fig. 1 as a continuous-space filter followed by ideal sampling on . The problem dealt with in this paper is, given the still LR image , obtain the best perceived higher resolution (HR) image defined on a denser sampling lattice . Here, we hypothesize that an ideal HR image defined on a denser lattice can be obtained in principle directly from by a virtual camera, which can similarly be modeled by filtering with a continuous-space filter followed by ideal sampling on . Our goal is then to obtain an estimate of denoted by with the highest perceptual quality. Fig. 1. Formulation of the image up-sampling problem based on models of the physical lower resolution camera and the theoretical higher resolution camera.