Dimensionality reduction approach for high dimensional text documents

Feature dimensionality has always been one of the key challenges in text mining as it increases complexity when mining documents with high dimensionality. High dimensionality introduces sparseness, noise, and boosts the computational and space complexities. Dimensionality reduction is usually addressed by implementing either feature reduction or feature selection techniques. In this work, the problem of dimensionality reduction is addressed by achieving feature reduction through the use of a novel membership function. For feature selection, Singular value decomposition and Information gain approaches are adopted through retaining top-k features. The approach of feature reduction is compared to feature selection techniques and results prove the dimensionality reduction achieved through proposed approach is better.