Predictive Distributed Learning based on Stability and Complexity Measures

Due to the inherent properties of high dimensional datasets, the majority of feature selection methods already in use do not scale well and eventually become inapplicable. As a result of this limitation, the performance of most classifiers are degraded. The objective of this paper is to propose an improved distributed feature selection technique by sample while maintaining class distribution towards obtaining optimal subsets using data complexity-stability measures. Whilst the Kuncheva and Jaccard were applied in obtaining stable subsets, the Fisher’s Discriminant Ratio was used to discriminate good candidate features from bad candidate features. The performance of the implementation framework was demonstrated using Brain Tumor Microarray datasets (using the original and the Synthetic Minority Oversampling Technique - induced datasets). The experiments contrasted the accuracy of three machine learning classifiers (decision tree, support vector machines and naive Bayes), trained with subsets of features generated by three selectors (information gain, gain ratio, and Chi-Squared). We observed that the highest accuracy percentages (100%, 100%, and 99.5%, respectively) were obtained by SVM, decision tree and naive Bayes respectively when the distributed learning techniques combined with data-complexity-stability measures were applied compared to existing techniques. Additionally, the stability of the feature subsets generated showed significant reduction both in the runtime complexities (2.97sec, 3.94sec, and 4.54sec), and the percentage of the number of selected features (0.52%, 1.39%, and 1.43%) with an alpha value of 0.75 respectively. These results showed that our proposed method achieved maximum performance and has future prospects.