Style Transfer by Relaxed Optimal Transport and Self-Similarity

The goal of style transfer algorithms is to render the content of one image using the style of another. We propose Style Transfer by Relaxed Optimal Transport and Self-Similarity (STROTSS), a new optimization-based style transfer algorithm. We extend our method to allow user specified point-to-point or region-to-region control over visual similarity between the style image and the output. Such guidance can be used to either achieve a particular visual effect or correct errors made by unconstrained style transfer. In order to quantitatively compare our method to prior work, we conduct a large-scale user study designed to assess the style-content tradeoff across settings in style transfer algorithms. Our results indicate that for any desired level of content preservation, our method provides higher quality stylization than prior work.

1 Introduction

One of the main challenges of style transfer is formalizing ’content’ and ’style’, terms which evoke strong intuitions but are hard to even define semantically. We propose formulations of each term which are novel in the domain of style transfer, but have a long history of successful application in computer vision more broadly. We hope that related efforts to refine definitions of both style and content will eventually lead to more robust recognition systems, but in this work we solely focus on their utility for style transfer.