Deep Neural Network-based Chinese Natural Language Generation on Dynamic Auxiliary Words

Many tasks have been solved by deep neural networks in the field of natural language processing (NLP) and natural language generation (NLG). However, some critical issues in teaching Chinese as a foreign or second language for foreign students are still unsolved. The teaching and learning of dynamic auxiliary words (le, zhe and guo) in Chinese are seldom considered in deep learning-powered applications. Taking three typical dynamic auxiliary words in Chinese as inputs, three Chinese language generation models are designed and evaluated in our work. The designed models, including BERT, BERT-WWM and RoBERTa, are optimized by observing the characteristics of two typical Chinese corpora. The experimental results show that BERT-WWM achieves better accuracy than RoBERTa and BERT by 29.4% and 3.983% on average, respectively. We also summarized the major error patterns on dynamic auxiliary word zhe based on detailed observations on falsely generated samples.