A novel algorithm for automatic document clustering

Internet has become an indispensible part of today's life. World Wide Web (WWW) is the largest shared information source. Finding relevant information on the WWW is challenging. To respond to a user query, it is difficult to search through the large number of returned documents with the presence of today's search engines. There is a need to organize a large set of documents into categories through clustering. The documents can be a user query or simply a collection of documents. Document clustering is the task of combining a set of documents into clusters so that intra cluster documents are similar to each other than inter cluster documents. Partitioning and Hierarchical algorithms are commonly used for document clustering. Existing partitioning algorithms have the limitation that the number of clusters has to be given as input and the clustering result depends on this input. If the number of clusters is not known, results are not acceptable. In this paper, we have developed a novel algorithm which generates number of clusters automatically for any unknown text dataset and clusters the documents appropriately based on Cosine Similarity between them. We have also detected zero clustering issue in partitioning algorithm and solved it using our novel algorithm.