A Novel Scalable Kernelized Fuzzy Clustering Algorithms Based on In-Memory Computation for Handling Big Data

Traditional scalable clustering algorithms mainly deal with the clustering of linearly separable data, but it is challenging to cluster the non-linear separable data efficiently in the feature space. In this article, we propose a novel Kernelized Scalable Random Sampling with Iterative Optimization Fuzzy c-Means (KSRSIO-FCM) clustering algorithm using Big Data framework. To propose the KSRSIO-FCM, we also propose the Kernelized version of Scalable Literal Fuzzy c-Means (KSLFCM) clustering algorithm, which is an integral part of the proposed KSRSIO-FCM algorithm. These kernelized clustering algorithms are evolved to deal with the non-linear separable problems by applying a kernel Radial Basis Functions (RBF) which maps the input data space non-linearly into a high dimensional feature space. We aim to design and implement the kernelized fuzzy clustering algorithms on Apache Spark, which performs the efficient clustering of Big Data due to its in-memory cluster computing technique. Exhaustive experiments are performed on various big datasets to show the effectiveness of proposed KSRSIO-FCM in comparison with other scalable clustering algorithms, i.e., KSLFCM, SRSIO-FCM, and SLFCM. The reported experimental results show that the KSRSIO-FCM algorithm in comparison with KSLFCM, SRSIO-FCM, and SLFCM achieves significant improvement in terms of time and space complexity, Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F-score, respectively. Furthermore, we have carried out a performance analysis of KSRSIO-FCM versus KSLFCM. Thus, the reported results show that the KSRSIO-FCM implemented on Apache Spark has better potential for Big Data clustering as compared to traditional scalable fuzzy clustering methods.

I. Introduction

Data has become an integral part of our life. There are 2.5 quintillion bytes of data at the current pace is generated every day. Many types of software are being designed to handle the voluminous data. Such an immense quantity of information containing valuable data is called Big Data. Clustering [1] is a data mining strategy that is extensively taken into consideration for mining significant information underlining unlabeled data. It is an unsupervised learning approach which applied on data to form a group such that data samples within a group share a similarity, this results in finding patterns of datasets [2]. In fuzzy clustering, a data sample with different degrees of membership can belong to various clusters.