Exploring Constant Signal Propagation to Optimize Neural Network Circuits

Deep neural networks tend to be extensively power and area consuming for their processing hardware. Integrated circuit designers face a considerable challenge when implementing machine learning hardware since DNNs require so many resources. This work focuses on optimizing circuits representing DNN in the form of AND-Inverter Graphs through approximate computing strategy. The optimization is done by analyzing the DNN training set to find constant or nearly constant values among the AIG nodes. The constant values are then propagated through the AIG, which results in removing unnecessary nodes. The trade-off between accuracy and its reduction in costs due to constant propagation is investigated. Experimental results show a significant reduction in circuit size with a slight loss in accuracy. For an AIG neural network that classifies the MNIST dataset, we achieved reductions of 29.4 % in size and 33.2 % in depth while losing only 1 % accuracy.