Loading [MathJax]/extensions/MathMenu.js
Energy-efficient ConvNets through approximate computing | IEEE Conference Publication | IEEE Xplore

Energy-efficient ConvNets through approximate computing


Abstract:

Recently convolutional neural networks (ConvNets) have come up as state-of-the-art classification and detection algorithms, achieving near-human performance in visual det...Show More

Abstract:

Recently convolutional neural networks (ConvNets) have come up as state-of-the-art classification and detection algorithms, achieving near-human performance in visual detection. However, ConvNet algorithms are typically very computation and memory intensive. In order to be able to embed ConvNet-based classification into wearable platforms and embedded systems such as smartphones or ubiquitous electronics for the internet-of-things, their energy consumption should be reduced drastically. This paper proposes methods based on approximate computing to reduce energy consumption in state-of-the-art ConvNet accelerators. By combining techniques both at the system- and circuit level, we can gain energy in the systems arithmetic: up to 30× without losing classification accuracy and more than 100× at 99% classification accuracy, compared to the commonly used 16-bit fixed point number format.
Date of Conference: 07-10 March 2016
Date Added to IEEE Xplore: 26 May 2016
ISBN Information:
Conference Location: Lake Placid, NY, USA
Citations are not available for this document.

1. Introduction

Recently neural networks have made an impressive comeback in the field of machine learning. Convolutional neural networks or ConvNets are consistently pushing the state-of-the-art in areas like computer vision and speech processing. One of the reasons for this revival is the increasing availability of computing power. Multicore CPU's, GPU's, and even clusters of GPU's are no longer prohibitively expensive and make it possible to train and evaluate larger networks.

Cites in Papers - |

Cites in Papers - IEEE (54)

Select All
1.
Zeke Raphael Lim Sy, Shihao Luo, Jean Flaherty, Truong Cong Thang, "Towards Green NeRF: an Exploration of Energy Influence Factors in NeRF Models", 2024 IEEE International Conference on Internet of Things and Intelligence Systems (IoTaIS), pp.65-71, 2024.
2.
Edward Manca, Luca Urbinati, Mario R. Casu, "STAR: Sum-Together/Apart Reconfigurable Multipliers for Precision-Scalable ML Workloads", 2024 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp.1-6, 2024.
3.
Edward Manca, Luca Urbinati, Mario R. Casu, "STAR: Sum-Together/Apart Reconfigurable Multipliers for Precision-Scalable ML Workloads", 2024 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp.1-6, 2024.
4.
Luca Urbinati, Mario R. Casu, "High-Level Design of Precision-Scalable DNN Accelerators Based on Sum-Together Multipliers", IEEE Access, vol.12, pp.44163-44189, 2024.
5.
Wouter Legiest, Furkan Turan, Michiel Van Beirendonck, Jan-Pieter D'Anvers, Ingrid Verbauwhede, "Neural Network Quantisation for Faster Homomorphic Encryption", 2023 IEEE 29th International Symposium on On-Line Testing and Robust System Design (IOLTS), pp.1-3, 2023.
6.
Luca Urbinati, Mario R. Casu, "Design-Space Exploration of Mixed-precision DNN Accelerators based on Sum-Together Multipliers", 2023 18th Conference on Ph.D Research in Microelectronics and Electronics (PRIME), pp.377-380, 2023.
7.
Shriram Shanbhag, Sridhar Chimalakonda, "An Exploratory Study on Energy Consumption of Dataframe Processing Libraries", 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR), pp.284-295, 2023.
8.
Song-Nien Tang, "Area-Efficient Parallel Multiplication Units for CNN Accelerators With Output Channel Parallelization", IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol.31, no.3, pp.406-410, 2023.
9.
Huruy Tesfai, Hani Saleh, Mahmoud Al-Qutayri, Moath B. Mohammad, Temesghen Tekeste, Ahsan Khandoker, Baker Mohammad, "Lightweight Shufflenet Based CNN for Arrhythmia Classification", IEEE Access, vol.10, pp.111842-111854, 2022.
10.
Eman Youssef, Hamed A. Elsimary, Magdy A. El-Moursy, Hassan Mostafa, Ahmed Khattab, "Energy-Efficient Precision-Scaled CNN Implementation With Dynamic Partial Reconfiguration", IEEE Access, vol.10, pp.95571-95584, 2022.
11.
Andres Rios, Patricia Nava, "Hardware for Quantized Mixed-Precision Deep Neural Networks", 2022 IEEE 15th Dallas Circuit And System Conference (DCAS), pp.1-5, 2022.
12.
Luca Urbinati, Mario R. Casu, "A Reconfigurable Depth-Wise Convolution Module for Heterogeneously Quantized DNNs", 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pp.128-132, 2022.
13.
Tianming Zhao, Yucheng Xie, Yan Wang, Jerry Cheng, Xiaonan Guo, Bin Hu, Yingying Chen, "A Survey of Deep Learning on Mobile Devices: Applications, Optimizations, Challenges, and Research Opportunities", Proceedings of the IEEE, vol.110, no.3, pp.334-354, 2022.
14.
Yue Chen, Huimin Du, Libo Chang, "A Reconfigurable micro-Processing Element for Mixed Precision CNNs", 2022 14th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA), pp.1-5, 2022.
15.
Ehab M. Ibrahim, Linyan Mei, Marian Verhelst, "Taxonomy and Benchmarking of Precision-Scalable MAC Arrays Under Enhanced DNN Dataflow Representation", IEEE Transactions on Circuits and Systems I: Regular Papers, vol.69, no.5, pp.2013-2024, 2022.
16.
Rachel Sterneck, Abhishek Moitra, Priyadarshini Panda, "Noise Sensitivity-Based Energy Efficient and Robust Adversary Detection in Neural Networks", IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol.41, no.5, pp.1423-1435, 2022.
17.
Antonio Cipolletta, Valentino Peluso, Andrea Calimera, Matteo Poggi, Fabio Tosi, Filippo Aleotti, Stefano Mattoccia, "Energy-Quality Scalable Monocular Depth Estimation on Low-Power CPUs", IEEE Internet of Things Journal, vol.9, no.1, pp.25-36, 2022.
18.
Ilaria Scarabottolo, Giovanni Ansaloni, George A. Constantinides, Laura Pozzi, "A Formal Framework for Maximum Error Estimation in Approximate Logic Synthesis", IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol.41, no.4, pp.840-853, 2022.
19.
Shen-Fu Hsiao, Yu-Che Yen, "Quantization of Deep Neural Network Models Considering Per-Layer Computation Complexity for Efficient Execution in Multi-Precision Accelerators", 2021 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), pp.1-2, 2021.
20.
Cecilia De la Parra, Ahmed El-Yamany, Taha Soliman, Akash Kumar, Norbert Wehn, Andre Guntoro, "Exploiting Resiliency for Kernel-Wise CNN Approximation Enabled by Adaptive Hardware Design", 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pp.1-5, 2021.
21.
Alice Sokolova, Mohsen Imani, Andrew Huang, Ricardo Garcia, Justin Morris, Tajana Rosing, Baris Aksanli, "MACcelerator: Approximate Arithmetic Unit for Computational Acceleration", 2021 22nd International Symposium on Quality Electronic Design (ISQED), pp.444-449, 2021.
22.
Yifan He, Jinshan Yue, Yongpan Liu, Huazhong Yang, "Block-Circulant Neural Network Accelerator Featuring Fine-Grained Frequency-Domain Quantization and Reconfigurable FFT Modules", 2021 26th Asia and South Pacific Design Automation Conference (ASP-DAC), pp.813-818, 2021.
23.
Sitao Huang, Aayush Ankit, Plinio Silveira, Rodrigo Antunes, Sai Rahul Chalamalasetti, Izzat El Hajj, Dong Eun Kim, Glaucimar Aguiar, Pedro Bruel, Sergey Serebryakov, Cong Xu, Can Li, Paolo Faraboschi, John Paul Strachan, Deming Chen, Kaushik Roy, Wen-mei Hwu, Dejan Milojicic, "Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators", 2021 26th Asia and South Pacific Design Automation Conference (ASP-DAC), pp.372-377, 2021.
24.
Eman Youssef, Hamed A. Elsemary, Magdy A. El-Moursy, Ahmed Khattab, Hassan Mostafa, "Energy Adaptive Convolution Neural Network Using Dynamic Partial Reconfiguration", 2020 IEEE 63rd International Midwest Symposium on Circuits and Systems (MWSCAS), pp.325-328, 2020.
25.
Shree Krishna Sharma, Xianbin Wang, "Toward Massive Machine Type Communications in Ultra-Dense Cellular IoT Networks: Current Issues and Machine Learning-Assisted Solutions", IEEE Communications Surveys & Tutorials, vol.22, no.1, pp.426-471, 2020.
26.
Mario Osta, Mohamad Alameh, Hamoud Younes, Ali Ibrahim, Maurizio Valle, "Energy Efficient Implementation of Machine Learning Algorithms on Hardware Platforms", 2019 26th IEEE International Conference on Electronics, Circuits and Systems (ICECS), pp.21-24, 2019.
27.
Daniele Jahier Pagliari, Enrico Macii, Massimo Poncino, "Automated Synthesis of Energy-Efficient Reconfigurable-Precision Circuits", IEEE Access, vol.7, pp.172030-172044, 2019.
28.
Seungsik Moon, Younghoon Byun, Jongmin Park, Sunggu Lee, Youngjoo Lee, "Memory-Reduced Network Stacking for Edge-Level CNN Architecture With Structured Weight Pruning", IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol.9, no.4, pp.735-746, 2019.
29.
Vincent Camus, Linyan Mei, Christian Enz, Marian Verhelst, "Review and Benchmarking of Precision-Scalable Multiply-Accumulate Unit Architectures for Embedded Neural-Network Processing", IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol.9, no.4, pp.697-711, 2019.
30.
Matteo Grimaldi, Valentino Peluso, Andrea Calimera, "Optimality Assessment of Memory-Bounded ConvNets Deployed on Resource-Constrained RISC Cores", IEEE Access, vol.7, pp.152599-152611, 2019.

Cites in Papers - Other Publishers (22)

1.
Mehwish Raza, Sharjeel Javed, Majida Kazmi, Arshad Aziz, Muhammad Fahim Ul Haque, Saad Ahmed Qazi, "Approximate Computing: Hardware and Software Techniques, Tools and Their Applications", Journal of Circuits, Systems and Computers, vol.33, no.04, 2024.
2.
Shiyu Wang, Shengbing Zhang, Xiaoping Huang, Libo Chang, "A high-efficiency spaceborne processor for hybrid neural networks", Neurocomputing, vol.541, pp.126230, 2023.
3.
Kunwer Mrityunjay Singh, Jatindra Deka, Santosh Biswas, "Incomplete Testing of SOC", Journal of Electronic Testing, 2023.
4.
Zihan Xia, Rui Wan, Jienan Chen, Runsheng Wang, "Reconfigurable spatial-parallel stochastic computing for accelerating sparse convolutional neural networks", Science China Information Sciences, vol.66, no.6, 2023.
5.
Kashif Inayat, Fahad Bin Muslim, Javed Iqbal, Syed Agha Hassnain Mohsan, Hend Khalid Alkahtani, Samih M. Mostafa, "Power-Intent Systolic Array Using Modified Parallel Multiplier for Machine Learning Acceleration", Sensors, vol.23, no.9, pp.4297, 2023.
6.
Tianwei Yan, Ning Zhang, Jie Li, Wenchao Liu, He Chen, "Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application", Remote Sensing, vol.14, no.13, pp.3130, 2022.
7.
Gert Goossens, Dirk Lanneer, Johan Van Praet, Werner Geurts, "Retargetable Compilation", Handbook of Computer Architecture, pp.1, 2022.
8.
Ahmed Badar, Arnav Varma, Adrian Staniec, Mahmoud Gamal, Omar Magdy, Haris Iqbal, Elahe Arani, Bahram Zonooz, "Highlighting the Importance of Reducing Research Bias and Carbon Emissions in CNNs", AIxIA 2021 ? Advances in Artificial Intelligence, vol.13196, pp.515, 2022.
9.
Yifan He, Jinshan Yue, Yongpan Liu, Huazhong Yang, "Block-Circulant Neural Network Accelerator Featuring Fine-Grained Frequency-Domain Quantization and Reconfigurable FFT Modules", Proceedings of the 26th Asia and South Pacific Design Automation Conference, pp.813, 2021.
10.
Wen?Long Chin, Qinyu Zhang, Tao Jiang, "Low?complexity neuron for fixed?point artificial neural networks with ReLU activation function in energy?constrained wireless applications", IET Communications, vol.15, no.7, pp.917, 2021.
11.
Daniele Jahier Pagliari, Francesco Daghero, Massimo Poncino, "Sequence-To-Sequence Neural Networks Inference on Embedded Processors Using Dynamic Beam Search", Electronics, vol.9, no.2, pp.337, 2020.
12.
Priyadarshini Panda, "QUANOS", Proceedings of the ACM/IEEE International Symposium on Low Power Electronics and Design, pp.187, 2020.
13.
Bert Moons, Marian Verhelst, "DVAFS?Dynamic-Voltage-Accuracy-Frequency-Scaling Applied to Scalable Convolutional Neural Network Acceleration", System-Scenario-based Design Principles and Applications, pp.99, 2020.
14.
Valentino Peluso, Andrea Calimera, "Energy-Accuracy Scalable Deep Convolutional Neural Networks: A Pareto Analysis", VLSI-SoC: Design and Engineering of Electronics Systems Based on New Computing Paradigms, vol.561, pp.107, 2019.
15.
Bert Moons, Daniel Bankman, Marian Verhelst, "BINAREYE: Digital and Mixed-Signal Always-On Binary Neural Network Processing", Embedded Deep Learning, pp.153, 2019.
16.
Kota ANDO, Kodai UEYOSHI, Yuka OBA, Kazutoshi HIROSE, Ryota UEMATSU, Takumi KUDO, Masayuki IKEBE, Tetsuya ASAI, Shinya TAKAMAEDA-YAMAZAKI, Masato MOTOMURA, "Dither NN: Hardware/Algorithm Co-Design for Accurate Quantized Neural Networks", IEICE Transactions on Information and Systems, vol.E102.D, no.12, pp.2341, 2019.
17.
Bert Moons, Daniel Bankman, Marian Verhelst, Embedded Deep Learning, pp.89, 2019.
18.
Bert Moons, Daniel Bankman, Marian Verhelst, "Embedded Deep Neural Networks", Embedded Deep Learning, pp.1, 2019.
19.
Bert Moons, Daniel Bankman, Marian Verhelst, "Hardware-Algorithm Co-optimizations", Embedded Deep Learning, pp.55, 2019.
20.
F. M. Castro, N. Guil, M. J. Marin?Jimenez, J. Perez?Serrano, M. Ujaldon, "Energy?based tuning of convolutional neural networks on multi?GPUs", Concurrency and Computation: Practice and Experience, vol.31, no.21, 2019.
21.
Valentino Peluso, Andrea Calimera, "Scalable-Effort ConvNets for Multilevel Classification", 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp.1-8, 2018.
22.
Masaya Kibune, Michael G. Lee, Neural Information Processing, vol.10635, pp.315, 2017.

References

References is not available for this document.