An PWL Based Fast Efficient Approximate Sigmoid Circuit Design for Activation Function

Neural networks with nonlinear activation functions are widely used in image recognition, signal processing and other fields. However, nonlinear activation function such as sigmoid function is difficult to implement efficiently in hardware because it requires complex exponentiation operation and division operation. In this paper, a new hardware circuit implementation for sigmoid function based on piecewise linear approximation (PWL) logic architecture is proposed. The proposed circuits aim to reduce hardware complexity and improve operation speed. Additionally, we propose to adopt approximate addition to furtherly save hardware cost by introducing LOA and MAA adders. The proposed hardware architecture implementations are based on a 90nm CMOS process and the experimental results indicate that the proposed architectures achieve better performance compared with other advanced recent works. The proposed approximate sigmoid circuit was verified in the application of an emotion recognition model for SEED-IV database and the application of a handwritten digit recognition model for MNIST dataset, and the prediction accuracy of the models with the approximate sigmoid circuit is almost the same to the one with the standard sigmoid circuit.