CNN Inference Costs Estimation on Microcontrollers: the EST Primitive-based Model

Neural network inference on embedded devices will have an important industrial impact on our society. Embedded devices are ubiquitous in many fields, like human activity recognition or visual object detection. As a matter of fact, Convolutional Neural Networks (CNNs) are now the best modality to solve most of computer vision problems. Although, the accuracy offered by these algorithms has a cost: an important energy consumption, a high execution time, and a significant memory footprint. This cost is a major challenge to implement CNNs within embedded devices with limited computational power, memory space and energy available. This makes prior estimation about the impact of a CNN on a given microcontroller, a design key point before applying neural network compression techniques. We introduce the EST primitive-based model to estimate the impact of a CNN on a microcontroller, regarding the latency, the power consumption and the needed memory space. The target hardware is the STM32L496ZG with CPU ARM Cortex M4 running at 14 different frequencies. Our model shows an average estimation error of 13.66% on latency, 5.52% on power consumption and 2.09% on needed memory space.

I. Introduction

Deep learning within the Internet of Things (IoT) is a high-growing field of interest, at the intersection of machine learning and embedded systems. Recent International Data Corporation forecast estimates that 79.4 zettabytes (ZB) of data will be generated by IoT devices in 2025. Billions of these data are already used for decision-making applications. Such applications analyze signals from a wide range of sensors. They use machine learning algorithms and are deployed on Micro Controllers Units (MCUs), in an edge computing paradigm. Among them are visual object detection [1], predictive maintenance [2], or the well-known keyword spotting [3].