Extreme learning machine: a new learning scheme of feedforward neural networks

It is clear that the learning speed of feedforward neural networks is in general far slower than required and it has been a major bottleneck in their applications for past decades. Two key reasons behind may be: 1) the slow gradient-based learning algorithms are extensively used to train neural networks, and 2) all the parameters of the networks are tuned iteratively by using such learning algorithms. Unlike these traditional implementations, this paper proposes a new learning algorithm called extreme learning machine (ELM) for single-hidden layer feedforward neural networks (SLFNs) which randomly chooses the input weights and analytically determines the output weights of SLFNs. In theory, this algorithm tends to provide the best generalization performance at extremely fast learning speed. The experimental results based on real-world benchmarking function approximation and classification problems including large complex applications show that the new algorithm can produce best generalization performance in some cases and can learn much faster than traditional popular learning algorithms for feedforward neural networks.

I. Introduction

From a mathematical point of view, research on the approximation capabilities of feedforward neural networks has focused on two aspects: universal approximation on compact input sets and approximation in a finite set. Many researchers have explored the universal approximation capabilities of standard multi-layer feedforward neural networks[1], [2], [3]. In real applications, the neural networks are trained in finite training set. For function approximation in a finite training set, Huang and Babri[4] shows that a sigle-hidden layer feedforward neural network (SLFN) with at most hidden neurons and with almost any nonlinear activation function can learn distinct observations with zero error. It should be noted that the input weights (linking the input layer to the first hidden layer) need to be adjusted in all these previous theoretical research works as well as in almost all practical learning algorithms of feedforward neural networks.