Recurrent Affine Transformation for Text-to-Image Synthesis

Text-to-image synthesis aims to generate realistic images conditioned on text descriptions. Recently, conditional affine transformations (CATs), such as conditional batch normalization and instance normalization, have been applied to different layers to control the contents in synthesized images. However, the isolated CAT blocks predict the batch statistics of neighboring layers independently. What's more, CATs are simple multilayer perceptions that are hard to optimize. To address above issues, we propose a recurrent affine transformation (RAT) that connects all the CAT blocks with a recurrent neural network for modeling the long-term dependency between CAT blocks. To verify the effectiveness of RAT, we conduct both microscopic and macroscopic analyses of RAT, which not only demonstrates the effectiveness of RAT but also turns out to be a useful perspective to analyze how GANs fuse conditional information. In addition, we apply a spatial attention mechanism to the discriminator, which helps the text description to supervise the generator to synthesize more relevant image contents. Extensive experiments on the CUB, Oxford-102, and COCO datasets demonstrate the proposed model's superiority in comparison to state-of-the-art models.