Abstract:
It is widely believed that the back-propagation algorithm in neural networks, for tasks such as pattern classification, overcomes the limitations of the perceptron. The a...Show MoreMetadata
Abstract:
It is widely believed that the back-propagation algorithm in neural networks, for tasks such as pattern classification, overcomes the limitations of the perceptron. The authors construct several counterexamples to this belief. They also construct linearly separable examples which have a unique minimum which fails to separate two families of vectors, and a simple example with four two-dimensional vectors in a single-layer network showing local minima with a large basin of attraction. Thus, back-propagation is guaranteed to fail in the first example, and likely to fail in the second example. It is shown that even multilayered (hidden-layer) networks can also fail in this way to classify linearly separable problems. Since the authors' examples are all linearly separable, the perceptron would correctly classify them. The results disprove the presumption, made in recent years, that, barring local minima, back-propagation will find the best set of weights for a given problem.<>
Published in: IEEE Transactions on Circuits and Systems ( Volume: 36, Issue: 5, May 1989)
DOI: 10.1109/31.31314
References is not available for this document.
Select All
1.
E. F. Beckenbach and R. Bellman, Inequalities, Germany, Berlin:Springer-Verlag, 1961.
2.
M. Brady, R. Raghavan and J. Slawny, Proc. IEEE 2nd Int. Conf. on Neural Networks, vol. 1, pp. 649-656, 1988.
3.
Automata Networks in Computer Science, NJ, Princeton:Princeton Univ. Press, 1987.
4.
G. E. Hinton and T. J. Sejnowski, "Optimal perceptual inference", Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition, pp. 448-453, 1983.
5.
M. Minsky and S. Papert, Perceptrons, MA, Cambridge:MIT Press, 1988.
6.
F. Rosenblatt, Principles of Neurodynamics, New York:Spartan, 1962.
7.
"8" in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, MA, Cambridge:MIT Press, vol. 1, 1986.
8.
J. Sklansky and G. N. Wassel, Pattern Classifiers and Trainable Machines, New York:Springer-Verlag, 1981.
9.
B. Wittner and J. Denker, "Strategies for teaching layered network classification tasks", Proc. Amer. Inst. Physics Conf. on Neural Networks, pp. 850-859, 1987-Nov.
10.
D. Zipser and R. A. Andersen, "A black-propagation programmed network that simulates response properties of a subset of posterior parietal neurons", Nature, vol. 331, no. 25, pp. 679-684, 1988.
