An Empirical Study of Convolutional Neural Network Compressions within Low-Power Devices

The unprecedented growth of Convolutional Neural Networks (CNNs) opens a new era of vision-based applications. The popularity of the Internet of Things (IoT), based on microcontroller units (MCUs), requires fully autonomous applications. While cloud-based solutions are the standard, implementing deep learning algorithms within IoT devices will improve data privacy and reduce energy consumption. However, the high inference cost of CNNs and the limited computational resources available on MCUs create a challenge. To overcome these limitations, the use of compression techniques is mandatory. These techniques aim at reducing the inference cost of the CNN while preserving the accuracy. The resulting compression rate is always underlined, but a comprehensive analysis of its impact is also required. To this end, we conducted an empirical study on compressed CNNs within MCUs to understand the real benefits of these compression techniques. Our study investigates the trade-off between the most common applied compression techniques and the inference cost of the reduced CNN.