Arbitrary Style Transfer With Style-Attentional Networks

Arbitrary style transfer aims to synthesize a content image with the style of an image to create a third image that has never been seen before. Recent arbitrary style transfer algorithms find it challenging to balance the content structure and the style patterns. Moreover, simultaneously maintaining the global and local style patterns is difficult due to the patch-based mechanism. In this paper, we introduce a novel style-attentional network (SANet) that efficiently and flexibly integrates the local style patterns according to the semantic spatial distribution of the content image. A new identity loss function and multi-level feature embeddings enable our SANet and decoder to preserve the content structure as much as possible while enriching the style patterns. Experimental results demonstrate that our algorithm synthesizes stylized images in real-time that are higher in quality than those produced by the state-of-the-art algorithms.

1. Introduction

Artistic style transfer is a technique used to create art by synthesizing global and local style patterns from a given style image evenly over a content image while maintaining its original structure. Recently, the seminal work of Gatys et al. [5] showed that the correlation between features extracted from a pre-trained deep neural network can capture the style patterns well. The method by Gatys et al. [5] is flexible enough to combine the content and style of arbitrary images, but is prohibitively slow due to the iterative optimization process.