Generating Artistic Images Via Few-Shot Style Transfer

Generating images from a predefined style with heterogeneous and limited data is a challenging task for generative models. This work focuses on the conditional generation of artistic images, aiming to learn from a small set of paintings with high variability how to convert real-world photos into impressionistic paintings with the same given style. We design a few-shot style transfer model using a mixture of diverse one-shot style transfer generative models based on the SinGAN model. The proposed few-shot model coineEnSinGAN utilizes an ensemble of different SinGAN realizations to style transfer realistic photos to their closest painting style, by incorporating a novel aggregation mechanism based on the minimum cosine distance in the latent space of the feature vectors. EnSinGAN generates convincing impressionistic landscape images, and was awarded the first place in the Kaggle competition “I’m something of a painter myself” by being the closest in distribution to the test images.