Error-Resilient Inference with an Error-Aware Activation Function in a Deep Neural Network

In this paper, we investigate impact of a bit-flip error on the accuracy of a deep neural network (DNN) model. Using MNIST as the application, we show that some bits, when flipped, can significantly degrade the accuracy of the model and thus requires additional methods for fault tolerance. Then, we propose an “error-aware activation” scheme which can tolerate errors that occur in a memory subsystem where deep learning parameters (weights and biases) reside. The proposed activation scheme increases average model accuracy from 75% to 99% and improves the worst-case model accuracy from 15% to 96% significantly.