Pico-Programmable Neurons to Reduce Computations for Deep Neural Network Accelerators

Deep neural networks (DNNs) have shown impressive success in various fields. As a response to the ever-growing precision demand of DNN applications, more complex computational models are created. The growing computational volume has become a challenge for the power and performance efficiency of DNN accelerators. This article presents a new neural architecture to prevent ineffective and redundant computations by using neurons with memory that have decision-making power. In addition, another local memory is used to keep calculation history for removing redundancy by computational reuse. Sparse computing, as another feature, is supported to remove computations of not only zero weights but also zero bits of each weight. The results on conventional datasets such as IMAGENET show a computational reduction of more than
$18 \times -150 \times $
. This scalable architecture enables 124 GOPS by using 197-mW power.