Toward Automatic Generation of Transcript From Spoken Lectures: The “Dream of the Red Chamber” Series

We present a corpus of Mandarin literature lectures. The recording device, environment, and the topics covered in the lectures are described briefly; then, we developed speech recognition and text simplification approaches for automatic transcription of the lectures. Given that the size of this corpus is relatively small, we applied transfer learning on AISHELL-1 by varying the number of transferred layers and fine-tuning the learning rates to improve the accuracy of speech recognition. Experimental results showed that a character error rate (CER) of 15.83% could be obtained. Additionally, by reducing the perplexity of the language model and the number of out-of-vocabulary words, the CER improved by another 0.29%. Further, to improve the fluency of transcription, we chose to deal with punctuation and pleonasm. Accuracy of 81% and 91.5% were reported by professional judges on adding punctuation and deleting pleonasm, respectively.