Model validation and determination for neural network activation function modeling

The unavailability of a robust model for actual physical activation functions has been the main obstacle to effectively training a VLSI implementation of a neural network. To deal with this problem, we have proposed a method for the training of a programmable neural network based on neuron modeling using in-the-loop data. In this paper, an analysis from a statistical perspective is presented which is targeted at solving two problems (a) Is a small neural network model structure sufficient to describe the physical nonlinear activation function? (b) Does the model meet the parsimony conditions? Our experimental results indicate that the method based on using a small neural network to model a physical neuron is practical and advantageous.