Hardware Implementation of Tanh Exponential Activation Function using FPGA

The most active research area for Field Programmable Gate Arrays is the Convolution Neural Network (CNN), and the gist of any CNN is an activation function. Therefore, various non-linear activation functions are required for deeper CNN. In this paper, we aim to implement the Tanh Exponential (TanhExp) activation function on Artix-7 and Zynq-7000. To this end, we will use the piecewise linear approximation and the second-order polynomial approximation while using the IEEE754 2008 floating-point representation. We present an investigation of the required hardware resources. We also evaluate the efficiency of each method of approximation and its derivative.