Sketch Beautification: Learning Part Beautification and Structure Refinement for Sketches of Man-Made Objects

We present a novel freehand sketch beautification method, which takes as input a freely drawn sketch of a man-made object and automatically beautifies it both geometrically and structurally. Beautifying a sketch is challenging because of its highly abstract and heavily diverse drawing manner. Existing methods are usually confined to their limited training samples and thus cannot beautify freely drawn sketches with both geometric and structural variations. To address this challenge, we adopt a divide-and-combine strategy. Specifically, we first parse an input sketch into semantic components, beautify individual components by a learned part beautification module based on part-level implicit manifolds, and then reassemble the beautified components through a structure beautification module. With this strategy, our method can go beyond the training samples and handle novel freehand sketches. We demonstrate the effectiveness of our system with extensive experiments and a perceptual study.

I. Introduction

Sketching is a universal and intuitive tool for humans to render and interpret the visual world. Even if sketches are usually drawn in an imprecise and abstract format, human viewers can still implicitly beautify them and easily envision their underlying objects. But for machines, existing computer algorithms [1], [2] are struggling to make use of these freely drawn sketches directly, in particular, the sketches created for depicting man-made objects with diverse geometry and non-trivial topology. Although the beautification problem has been studied for decades, ranging from the primitives of sketched geometric objects [3] to strokes in handwritings [4], systematic analysis of beautifying sketches for man-made objects has rarely been studied. The key challenge here is how to instantiate poorly drawn conceptual geometries and refine imprecise structures simultaneously. Addressing the beautification problem of man-made object sketches can inspire and facilitate various downstream sketch-based applications such as sketch-based modeling [1], [5], [6], sketch-based retrieval [7], [8], and other sketch understanding tasks [9].