A 0.23mW Heterogeneous Deep-Learning Processor Supporting Dynamic Execution of Conditional Neural Networks

A deep-learning processor is presented for achieving ultra-low-power operation in mobile applications. Using a heterogeneous architecture that includes a low-power always-on front-end and a selectively-enabled high-performance backend, the processor dynamically adjusts computational resources at runtime to support conditional execution in neural networks and meet performance targets with increased energy efficiency. Featuring a reconfigurable datapath and a memory architecture optimized for energy efficiency, the processor supports multilevel dynamic activation of neural network segments, performing object detection tasks with 5.3× lower energy consumption in comparison with a static baseline design. Fabricated in 40nm CMOS, the processor test-chip dissipates 0.23m W at 5.3 fps. It demonstrates energy scalability up to 28.6 TOPS/W and can be configured to run a variety of workloads, including severely-power-constrained ones such as always-on monitoring in mobile applications.

I. Introduction

Deep neural networks (DNNs) are essential elements of Artificial Intelligence (AI) systems. With AI capabilities increasingly moving from data centers to embedded and mobile platforms, there is growing need for energy-efficient DNN hardware designs capable of supporting always-on operation while meeting stringent power constraints.