Auto-LUT: Auto Approximation of Non-Linear Operations for Neural Networks on FPGA

The approximation of non-linear operation can simplify the logic design and save the system resources during the neural network inference on Field-Programmable Gate Array (FPGA). Prior work can approximate the non-linear operations with piecewise linear (PWL) function, but such approximation neglects considering the hardware overhead simultaneously. This paper proposes a novel approximation framework called Auto-LUT, which leverages a neural network to automatically approximate the non-linear operations. The framework formulates the approximation error and hardware overhead as a multi-objective optimization problem and employs an automated search mechanism to find the minimum number of segments and data bit width. To improve the approximation accuracy, we propose a bias clipping operation during the training of approximation networks, which enforces the model to approximate within the range of interest. Moreover, a hardware-friendly quantization scheme is further introduced to simulate the hardware behavior, thereby reducing the hardware overhead. Finally, a customized hardware architecture based on FPGA is utilized to deploy the quantized result. The experimental results show that Auto-LUT costs 56.32% less LUTs and 32.31% less flip-flops (FF) while reducing 4.32% approximation error compared to the state-of-the-art method.