ComboGAN: Unrestrained Scalability for Image Domain Translation

The past year alone has seen unprecedented leaps in the area of learning-based image translation, namely Cycle-GAN, by Zhu et al. But experiments so far have been tailored to merely two domains at a time, and scaling them to more would require an quadratic number of models to be trained. And with two-domain models taking days to train on current hardware, the number of domains quickly becomes limited by the time and resources required to process them. In this paper, we propose a multi-component image translation model and training scheme which scales linearly - both in resource consumption and time required - with the number of domains. We demonstrate its capabilities on a dataset of paintings by 14 different artists and on images of the four different seasons in the Alps. Note that 14 data groups would need (14 choose 2) = 91 different CycleGAN models: a total of 182 generator/discriminator pairs; whereas our model requires only 14 generator/discriminator pairs.

1. Introduction

In such a short amount of time, we have come such a long way in the field of image domain adaptation and style transfer, with projects such as [12], [9], [5], [6], [10], [3]–, and more paving the way. The first four are of particular interest as they do not simply transfer style in terms of texture and color, but in terms of semantics, and they maintain realism in their results. But they must be trained on examples from two specific domains, whereas the other two do not. The first three are even more noteworthy for being able to do this without any supervision between the two domains chosen - no pairing of matching data. Now what if we want to go beyond two domains?