PTR-MirrorGAN: Parallel Text Redescription module for advancing MirrorGAN

During generating images from a given text description, two aspects of consistency are usually considered: visual consistency and semantic consistency. Although there has been a great improvement in the use of generative adversarial networks to generate images, it remains a challenging project to ensure that the semantics of the generated images are consistent with the semantics of the input text descriptions. This study proposed a Real-Fake Images Parallel Text Redescription Module (RFIPTRM) for advancing MirrorGAN based on the idea of mirroring and reconstructed the loss of the image generation model by adding the loss between the real text and the text from generated images to the genarators for balancing the textual semantic space. Intuitively, there may be inconsistency between the textual semantic space structure of our real text and the generated image, so we conducted adversarial learning between redescriptive text from the real images and the real text to improve the inconsistency of this text space structure. Comprehensive experiments conducted on the public benchmark datasets CUB and COCO have proved that PTR-MirrorGAN is superior to other methods and achieved a better generation effect.