High-dimensional function approximation using local linear embedding

Neural network approximation of high-dimensional nonlinear functions is difficult due to the sparsity of the data in the high-dimensional data space and the need for good coverage of the data space by the `receptive fields' of the neurons. However, high-dimensional data often resides around a much lower dimensional supporting manifold. Given that a low dimensional approximation of the target function is likely to be more precise than a high-dimensional approximation, if we can find a mapping of the data points onto a lower-dimensional space corresponding to the supporting manifold, we expect to be able to build neural network approximations of the target function with improved precision and generalization ability. Here we use the local linear embedding (LLE) method to find the low-dimensional manifold and show that the neural networks trained on the transformed data achieve much better function approximation performance than neural networks trained on the original data.