A hardware-aware neural architecture search algorithm targeting low-end microcontrollers

Hardware-aware neural architecture search (HW NAS), the process of automating the design of neural architectures taking into consideration hardware constraints, has already outperformed the best human designs on many tasks. However, it is known to be highly demanding in terms of hardware, thus limiting access to non-habitual neural network users. Fostering its adoption for the next-generation IoT and wearable devices design, we propose an HW NAS that can be run on laptops, even if not mounting a GPU. The proposed technique, designed to have both a low search cost and resource usage, produces tiny convolutional neural networks (CNNs) targeting low-end microcontrollers. It achieves state-of-the-art results in the human-recognition tasks, on the Visual Wake Word dataset a standard TinyML benchmark, in just 3:37:0 hours on a laptop mounting an 11th Gen Intel(R) Core(TM) i7-11370H CPU @ 3. 30GHz equipped with 16 GB of RAM and 512 GB of SSD, without using a GPU.

I. Introduction

The outstanding success of convolutional neural networks (CNNs) for image classification has been supported by the increasing amount of computational power offered by GPUs. Indeed, in the recent years researchers have tried to reproduce such results on devices with limited computational complexity. Manually designed CNNs for mobile devices like Mobilenets [1], Shufflenets [2], and EfficientNets [3] have surged. Based on such efforts, projects like Mnasnet [4] and Fbnet [5] have tried to automatize the process of designing CNNs for custom tasks, given different hardware targets. These techniques gave birth to Hardware-aware neural architecture search (HW NAS).