A short text clustering study integrating context-awareness and contrast learning

Traditional text representation techniques will produce high-dimensional and sparse vectors for short texts since they have fewer words, which increases the likelihood of noise interference and lowers the accuracy of short text clustering. An alternative model, CASA-Bert-K, is investigated to overcome the aforementioned issues. First, In order to add contextual features into the text representation, context-awareness self-attention networks are combined with Bert. Secondly, To de-optimize the representation of short texts, a simple and effective data augmentation approach to contrast learning is introduced. The text representation module and the clustering module are jointly trained to optimize both text representation and clustering in order to further improve the clustering accuracy. The experimental findings show the researched model's effectiveness in raising the accuracy rate through the use of three datasets in the tests.