High-Performance and Low-Cost Approximation of ANN Sigmoid Activation Functions on FPGAs

Neural network implementations on FPGAs have received significant attention from the research community due to their superior performance in tasks such as computer vision and the need to perform them on edge devices. Most efforts have focused on architecture optimizations such as pruning and operations such as the multiplications and additions with less attention on the ANN activation function. In this paper a hardware implementation of an artificial neural network on Field Programmable Gate Arrays (FPGA) is presented. A digital system architecture is designed with the purpose of reducing the delay of implementation and minimizing the LUTs in the FPGA. The designed architecture is described using Very High-Speed Integrated Circuits Hardware Description Language (VHDL). The proposed method utilizes a known piecewise linear approximation of the sigmoid function, and additionally exploits properties of fixed-point binary number representation to efficiently reduce delay and the amount of LUTS required Compared with previously published implementations of the same function and FPGA technology, the proposed method is shown to achieve a reduction between 33% and 95% in LUTs with no reduction in accuracy.