A Novel 2-Step Very Large-Scale Short Text Clustering Algorithm

In recent research, the effectiveness of text clustering is the long-term pursuit goal, because the same clustering documents have the maximum correlation and internal connection, this technology can simplify the user's text processing process. This research study focuses on the very large-scale short text clustering with a case study of legal documents. First, the normal size short text clustering algorithm is studied, the encapsulation method is used, and the Term Frequency-Inverse Document Frequency (TFIDF) algorithm is selected. Then, after finding the shortcomings, the 2-step algorithm for very large short text clustering is proposed. First, the text similarity reduction serves as the pre-processing, and this study applies the iteration function together with the Simhash algorithm to achieve the goal. Second, the class-centric algorithm extension method is designed to effectively cluster massive information. The performance of the algorithm is evaluated by experiment in terms of time and accuracy.