Loading [MathJax]/extensions/MathZoom.js
Energy-efficient ConvNets through approximate computing | IEEE Conference Publication | IEEE Xplore

Energy-efficient ConvNets through approximate computing


Abstract:

Recently convolutional neural networks (ConvNets) have come up as state-of-the-art classification and detection algorithms, achieving near-human performance in visual det...Show More

Abstract:

Recently convolutional neural networks (ConvNets) have come up as state-of-the-art classification and detection algorithms, achieving near-human performance in visual detection. However, ConvNet algorithms are typically very computation and memory intensive. In order to be able to embed ConvNet-based classification into wearable platforms and embedded systems such as smartphones or ubiquitous electronics for the internet-of-things, their energy consumption should be reduced drastically. This paper proposes methods based on approximate computing to reduce energy consumption in state-of-the-art ConvNet accelerators. By combining techniques both at the system- and circuit level, we can gain energy in the systems arithmetic: up to 30× without losing classification accuracy and more than 100× at 99% classification accuracy, compared to the commonly used 16-bit fixed point number format.
Date of Conference: 07-10 March 2016
Date Added to IEEE Xplore: 26 May 2016
ISBN Information:
Conference Location: Lake Placid, NY, USA
References is not available for this document.

1. Introduction

Recently neural networks have made an impressive comeback in the field of machine learning. Convolutional neural networks or ConvNets are consistently pushing the state-of-the-art in areas like computer vision and speech processing. One of the reasons for this revival is the increasing availability of computing power. Multicore CPU's, GPU's, and even clusters of GPU's are no longer prohibitively expensive and make it possible to train and evaluate larger networks.

Select All
1.
T. Chen, Z. Du, N. Sun, J. Wang, C. Wu, Y. Chen, et al., "Diannao: A small-footprint high-throughput accelerator for ubiquitous machine-learning", Proceedings of the 19th international conference on Architectural support for programming languages and operating systems, pp. 269-284, 2014.
2.
F. Conti and L. Benini, "A ultra-low-energy convolution engine for fast brain-inspired vision in multicore clusters", Proceedings of the 2015 Design Automation & Test in Europe Conference & Exhibition EDA Consortium, pp. 683-688, 2015.
3.
M. de la Guia Solaz, W. Han and R. Conway, "A flexible low power dsp with a programmable truncated multiplier", Circuits and Systems I: Regular Papers IEEE Transactions, vol. 59, no. 11, pp. 2555-2568, 2012.
4.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database", IEEE Conference on Computer Vision and Pattern Recognition 2009. CVPR 2009, pp. 248-255, June 2009.
5.
G. Dundar and K. Rose, "The effects of quantization on multilayer neural networks", IEEE transactions on neural networks/a publication of the IEEE Neural Networks Council, vol. 6, no. 6, pp. 1446-1451, 1994.
6.
C. Farabet, B. Martini, B. Corda, P. Akselrod, E. Culurciello and Y. LeCun, "Neuflow: A runtime reconfigurable dataflow processor for vision", Computer Vision and Pattern Recognition Workshops (CVPRW) 2011 IEEE Computer Society Conference, pp. 109-116, 2011.
7.
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, et al., "Caffe: Convolutional Architecture for Fast Feature Embedding", arXiv:1408.5093 [cs], June 2014.
8.
M. Jiang and G. Gielen, "The effects of quantization on multilayer feedforward neural networks", International Journal of Pattern Recognition and Artificial Intelligence, vol. 17, no. 04, pp. 637-661, June 2003.
9.
A. Krizhevsky, "Learning multiple layers of features from tiny images", 2009.
10.
"ImageNet Classification with Deep Convolutional Neural Networks" in Advances in Neural Information Processing Systems, Curran Associates, Inc, vol. 25, pp. 1097-1105, 2012.
11.
P. Kulkarni, P. Gupta and M. Ercegovac, "Trading accuracy for power with an underdesigned multiplier architecture", VLSI Design (VLSI Design) 2011 24th International Conference, pp. 346-351, 2011.
12.
B. B. Le Cun, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard and L. D. Jackel, "Handwritten digit recognition with a back-propagation network", Advances in neural information processing systems. Citeseer, 1990.
13.
Y. LeCun and C. Cortes, "The mnist database of handwritten digits", 1998.
14.
B. Moons and M. Verhelst, "Dvas: Dynamic voltage accuracy scaling for increased energy-efficiency in approximate computing", International Symposium on Low Power Electronics and Design (ISLPED), 2015.
15.
M. Peemen, A. A. Setio, B. Mesman and H. Corporaal, "Memory-centric accelerator design for convolutional neural networks", Computer Design (ICCD) 2013 IEEE 31st International Conference, pp. 13-19, 2013.
16.
K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition", CoRR abs/1409.1556, 2014.
17.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, et al., "Going Deeper with Convolutions", arXiv:1409.4842 [cs], Sept. 2014.
18.
S. Venkataramani, V. K. Chippa, S. T. Chakradhar, K. Roy and A. Raghunathan, "Quality programmable vector processors for approximate computing", Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture, pp. 1-12, 2013.
19.
S. Venkataramani, A. Ranjan, K. Roy and A. Raghunathan, "Axnn: energy-efficient neuromorphic systems using approximate computing", Proceedings of the 2014 international symposium on Low power electronics and design, pp. 27-32, 2014.
20.
S. Venkataramani, A. Sabne, V. Kozhikkottu, K. Roy and A. Raghunathan, "Salsa: systematic logic synthesis of approximate circuits", Proceedings of the 49th Annual Design Automation Conference, pp. 796-801, 2012.
21.
R. Wu, S. Yan, Y. Shan, Q. Dang and G. Sun, "Deep Image: Scaling up Image Recognition", arXiv:1501.02876 [cs], Jan. 2015.

References

References is not available for this document.