Self-Adaptive Hybrid Extreme Learning Machine for Heterogeneous Neural Networks

This paper presents a hybrid algorithm for the creation of heterogeneous single layer neural networks (SLNNs). The proposed self-adaptive heterogeneous hybrid extreme learning machine (SA-He-HyELM) trains a series of SLNNs with different neuron types in the hidden layer utilizing the extreme learning machine (ELM) algorithm. These networks are evolved into heterogeneous networks (networks having different combinations of hidden neurons) with the help of a modified genetic algorithm (GA). The algorithm is able to handle two architecturally different neuron types: traditional low order (linear) units and higher order units with different transfer functions. The GA is fully self-adaptive and uses one novel hybrid crossover operator along with a self-adaptive mutation operator in order to retain ELM's simplicity and minimize the number of parameters need tuning. The experimental part of the current paper involves testing SA-He-HyELM with traditional ELM and other three ELM-based methods. The experimental part utilized a series of regression and classification experiments on relatively large datasets. In all cases the proposed method managed to get lower MSE or higher classification accuracy when compared to the aforementioned methods.