A Novel Distilled Generative Essay Polish System via Hierarchical Pre-Training

In language processing tasks, the most important process in automated text polishment always consists of text correction and text supplementation. Finding that text polishment is a necessary step in the field of English essay reviewing, we are motivated to be the first of building an end-to-end automated English essay polish system, to support writing instruction. There were independent methods for text correction tasks and text supplementation tasks, but when combining them for essay polishment tasks, conflicts arise from their interplay. In this paper, we propose a polish system that elegantly performs text correction and text supplementation at the same time, achieving an improved revision quality. Furthermore, we design a closed-loop essay polishing process, made up of a Rewriting Model and a Scoring Model, which refers to modified GPT2 and ensembled Bert respectively. The rewriting process targets the deficiencies of the essays by a threshold controlled mechanism. Lastly, the performance of our proposed system is further enhanced by an optimization method. Intensive experiments on both real data and simulated data have shown score improvements on full essays by Scoring Model, as well as higher text correction accuracy and longer text supplementation length.