Thai Language Sentence Segmentation Based on N-Gram Context Model

Segmentation of Thai language's sentences is the basic task of natural language processing of Thai language and has important significance on studying lexical analysis, syntactic analysis and machine translation relating to Thai language. Combined with statistical machine learning methods and features of Thai language, a segmentation method of Thai language's sentences based on n-gram context model was proposed. The Orchid1997 corpuses were expanded through this method based on the segmentation tasks of Thai language's sentences. And then, the symbols during Thai language's paragraphs (such as words and space, etc.) were marked using n-gram algorithm depend on the lexical features and syntactic features of Thai language. The segmentation of Thai language's sentences was implemented through identifying the space character at the end of the sentence. The experimental results show that this method can realize reliable segmentation of Thai language's sentences by utilized language characteristics effectively. The comprehensive segmentation performance of space and comprehensive performance F value of sentence segmentation is easy to reach 85.43% and 71.67%, respectively.