An Improved Self-Supervised Learning Method for Controllable Content Generation in Large Language Models

The widespread application of pretrained large language models for content generation has focused on overcoming the illusion of control in text generation. While fine-grained control at the word and phrase level allows for partially controlled text generation, the probability of producing content that aligns with human natural language patterns remains low. This article proposes an enhanced self-supervised learning approach to improve the quality of controllable text generation for this issue. Building upon the CoCon model, our approach ensures controllability in text generation at both the sentence and word granularity. It amplifies features in a high-dimensional space, filtering out irrelevant feature information to aid the model in generating more precise content. Experimental results on publicly available datasets demonstrate that our approach effectively reduces Perplexity, enhances model performance, and produces text that better conforms to natural language conventions.