Neural networks and approximation by superposition of Gaussians

The aim of this paper is to discuss a nonlinear approximation problem relevant to the approximation of data by radial-basis-function neural networks. The approximation is based on superpositions of translated Gaussians. The method used enables us to give explicit approximations and error bounds. New connections between this problem and sampling theory are exposed, but the method used departs radically from those commonly used to obtain sampling results since (i) it applies to signals that are not band-limited, and possibly even discontinuous; (ii) the sampling knots (the centers of the radial-basis functions) need not be equidistant; (iii) the basic approximation building block is the Gaussian, not the usual sinc kernel. The results given offer an answer to the following problem: how complex should a neural network be in order to be able to approximate a given signal to better than a certain prescribed accuracy? The results show that O(1/N) accuracy is possible with a network of N basis functions.