Implementation of Activation functions using inverse delta coding LUT

Activation functions are essential components of deep neural networks (DNNs). However, the use of exponential functions and dividers for non-linearity significantly increases hardware costs. This article presents a hardware architecture for activation functions that uses inverse delta coding LUT. We implemented an inverse LUT approach by utilizing the fact that the output range is smaller than the input range. Additionally, to lower the bit-depth of LUT components, delta coding was applied, resulting in a 64.45% reduction in the size of the lookup table compared to full size LUT, with no significant decrease in accuracy.