PWL- Explorer: A Reconfigurable Architecture for Nonlinear Activation Function with Automatic DSE

Nonlinear functions are indispensable parts of deep neural networks (DNNs). Based on the piecewise approximation with the non-uniform segmentation strategy, this paper proposes PWL- Explorer, a highly parameterized reconfigurable nonlinear core designed with Chisel to implement diverse nonlinear functions. Additionally, we model the design space exploration (DSE) as a multi-objective black-box optimization problem via Bayesian optimization to explore the Pareto front of PWL- Explorer for the precision and area-delay product (ADP) objectives. The experimental results show that compared to state-of-the-art work for element-wise activation functions, the optimized PWL-Explorer architecture achieves an average of 1.58× better ADP while obtaining 2.93 × higher maximum absolute error (MAE) precision. Compared to state-of-the-art work for Softmax, PWL-Explorer achieves 1.05× better ADP while obtaining 40.01 x better mean square error (MSE) precision. Due to the elaborate hardware design and push-button workflow, PWL-Explorer can provide support to DNN accelerator developers.