Learn to Sketch: A fast approach for universal photo sketch

Rendering real photos to abstract sketches is an interesting application that can help us understand the key features. In this paper, we propose a universal photo sketch model via a deep convolutional neural network. Prior arts often cast this problem as an edge or contour detection. However, the edges or contours may not exactly reflect the boundaries of the contents of the photos. They also fail to reveal the occlusion that separates the objects from each other. We resolve this problem by proposing Photo2Sketch and Sketch2Photo to form a loop to bridge the gap between photos and sketches. We introduce relevant sketch references as indicators to supervise the sketch generation. Meanwhile, we also introduce an adaptive sketching process that can generate drawing with confidence, hence multiple sketches can be obtained. Experimental results show that our proposed method surpasses other state-of-the-art methods in both qualitative and quantitative measures.

I. Introduction

In computer vision or computer graphics, edge-like visual-ization is one of the interesting tools in image representation. It can be edges, textures, contours, boundaries, and so on. U sing edge-like visualization can reduce the data redundancy in image processing. Automatic edge-like generation can be useful for geometric understanding [7], object recognition [9], fashion design [14] and other applications. Sketch, in a general definition, is one type of edge representation of the image. Yet it is not just edges but connects related edges to form continuum boundaries for objects. It is not even just contours but preserves more detailed features to reveal the occlusion of objects. It differs from semantic segmentation, sketches can describe more detailed textures. Perceptually, it resembles how human observe the world. For art design, a sketch is an important artistic language that can reflect the rationality of visual perception and aesthetic appreciation.