Abstract:
In this paper, mapping networks will be considered from the viewpoint of the piecewise-linear (PWL) approximation. The so-called canonical representation plays a kernel r...Show MoreMetadata
Abstract:
In this paper, mapping networks will be considered from the viewpoint of the piecewise-linear (PWL) approximation. The so-called canonical representation plays a kernel role in the PWL representation theory. While this theory has been researched intensively in the contents of mathematics and circuit simulations, little has been seen in the research area about the theoretical aspect of neural networks. This paper modifies this theory and applies it as a mathematical support for mapping networks. The main modification is a "higher-level" generalization of the canonical representation with proofs of its availability in the set of PWL functions. The modified theory will first be used to study the canonical PWL feature of the popular multilayer perceptron-like (MLPL) networks. Second, it will be seen that the generalized canonical representation is itself suitable for a network implementation, which is called the standard canonical PWL network. More generally, the family of (generalized) canonical PWL networks is defined as those which may take the canonical PWL representation as a mathematical model. This family is large and practically meaningful. The standard canonical PWL networks may be taken as representatives in the family. The modification of the PWL representation theory as well as the introduction of this theory in the theoretical study of mapping networks, which provide a new concept of mapping networks, i.e., the canonical PWL network family, may be regarded as the main contributions of the paper.<>
Published in: IEEE Transactions on Neural Networks ( Volume: 6, Issue: 1, January 1995)
DOI: 10.1109/72.363451
References is not available for this document.
Select All
1.
F. Rosenblatt, Principles of Neurodynamics, New York:Spartan, 1962.
2.
D. E. Rumelhart, G. E. Hinton and R. J. Williams, "Learning representations by back-propagating errors", Nature, vol. 323, no. 533, 1986.
3.
B. Widrow, R. G. Winter and R. A. Baxter, "Layered neural nets for pattern recognition", IEEE Trans. Acoust. Speech Signal Process., vol. ASSP-36, pp. 1109-1118, 1988.
4.
W. S. McCulloch and W. Pitts, "A logical calculus of ideas immanent in nervous activity", Bull. Math. Biophys., vol. 5, pp. 115-133, 1943.
5.
K. Funahashi, "On the approximate realization of continuous mappings by neural networks", Neural Net., vol. 2, pp. 183-192, 1989.
6.
K. Hornik, M. Stinchcombe and H. White, "Multilayer feedforward networks are universal approximators", Neural Net., vol. 2, pp. 359-366, 1989.
7.
G. Cybenko, "Approximation by superposition of a sigmoidal function", Math. Control Signals and Syst., vol. 2, pp. 303-314, 1989.
8.
T. Poggio and F. Girosi, "Networks for approximation and learning", Proc. IEEE, vol. 78, pp. 1481-1497, 1990.
9.
E. K. Blum and L. K. Li, "Approximation theory and feedforward networks", Neural Net., vol. 4, pp. 511-515, 1991.
10.
S. Geva and J. Sitte, "A constructive method for multivariate function approximation by multilayer perceptrons", IEEE Trans. Neural Net., vol. 3, pp. 621-624, 1992.
11.
A. Lapedes and R. Farber, "How neural nets work" in Neural Information Processing Systems, New York:American Institute of Physics, pp. 442-456, 1988.
12.
J.-N. Lin and R. Unbehauen, "Adaptive nonlinear digital filter with canonical piecewise-linear structure", IEEE Trans. Circ. Syst., vol. 37, pp. 347-353, 1990.
13.
R. Batruni, "A multilayer neural network with piecewise-linear structure and back-propagation learning", IEEE Trans Neural Net., vol. 2, pp. 395-403, 1991.
14.
L. O. Chua and S. M. Kang, "Section-wise piecewise-linear functions: Canonical representation properties and applications", Proc. IEEE, vol. 65, pp. 915-929, 1977.
15.
C. de Boor, A Practical Guide to Splines, New York:Springer-Verlag, 1978.
16.
L. O. Chua and A. C. Deng, "Canonical piecewise-linear representation", IEEE Trans. Circ. Syst., vol. 35, pp. 101-111, 1988.
17.
C. Kahlert and L. O. Chua, "A generalized canonical piecewise-linear representation", IEEE Trans. Circ. Syst., vol. 37, pp. 373-383, Mar. 1990.
18.
C. Güzelis and I. C. Göknar, "A canonical representation for piecewise-affine maps and its applications to circuit analysis", IEEE Trans. Circ. Syst., vol. 38, pp. 1342-1354, 1991.
19.
C. Kahlert and L. O. Chua, "The complete canonical piecewise-linear representation—Part I: The geometry of the domain space", IEEE Trans. Circ. Syst. I, vol. 39, pp. 222-236, 1992.
20.
A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing, New York:Wiley, 1993.
21.
T. A. M. Kevenaar and D. M. W. Leenaerts, "A comparison of piecewise-linear model descriptions", IEEE Trans. Circ. Syst. I, vol. 39, pp. 996-1004, 1992.
22.
T. Ohtsuki, T. Fujisawa and S. Kumagai, "Existence theorem and a solution algorithm for piecewise-linear resistor networks", SIAM J. Math. Anal., vol. 8, pp. 69-99, 1977.
23.
J.-N. Lin and R. Unbehauen, "Canonical representation: From piecewise-linear function to piece wise-smooth functions", IEEE Trans. Circ. Syst. I, vol. 40, pp. 461-468, 1993.
24.
J.-N. Lin and R. Unbehauen, "A generalization of canonical piecewise-linear functions", IEEE Trans. Circ. Syst., vol. 41, pp. 345-347, 1994.
