Penalty-based Sequence Generative Adversarial Networks with Enhanced Transformer for Text Generation

In this paper, we propose a new model to solve the problem of text generation, which is based on the concept of seqGAN, combines self-attention with modeling localness, and introduces the penalty-based objective function. This model has much better performance than the original model. In the original model, the generator's ability of the text feature extraction is insufficient. We introduce self-attention with modeling localness, which greatly enhances its ability to capture long distance and shortrange dependencies. In addition, we use the penalty-based objective function instead of the loss function of the original model to solve the problem of mode collapse. Experimental results demonstrate that our model consistently outperforms several state-of-the-art text generation methods in the quality of generated texts.