Hybrid Algorithm of Close-Correlation Subset Extraction for Big Data

A composite scheme of subset extraction is here proposed upon data correlation for Big Data mining. Two stages of processing are applied to subset extraction. Four operations (conditional storage, data processing, similarity calculation and pattern learning) are designed, each of which corresponds to multiple objects. This paper mainly studies the extraction of closely related sets of Chinese vocabulary semantics. Based on the complexity of Chinese semantics, this paper maps Chinese vocabulary into English, extracts the eigenvalues of English vocabulary through vocabulary vectorization and principal component analysis (PCA). Then, it clusters the set of closely related items of English vocabulary by K-means algorithm. Finally, it builds the set of closely related items of Chinese vocabulary semantics based on this evidence. For the other information, it builds the set of closely related items of Chinese vocabulary semantics. Potential users create a closely related subset of corpus that is readily available. Sample applications are also provided to illustrate the main algorithms. The results show that the algorithm can effectively reduce the size of data and make the information available.