DiTMoS: Delving into Diverse Tiny-Model Selection on Microcontrollers

Enabling efficient and accurate deep neural network (DNN) inference on microcontrollers is non-trivial due to the constrained on-chip resources. Current methodologies primarily focus on compressing larger models yet at the expense of model accuracy. In this paper, we rethink the problem from the inverse perspective by constructing small/weak models directly and improving their accuracy. Thus, we introduce DiTMoS, a novel DNN training and inference framework with a selector-classifiers architecture, where the selector routes each input sample to the appropriate classifier for classification. DiTMoS is grounded on a key insight: a composition of weak models can exhibit high diversity and the union of them can significantly boost the accuracy upper bound. To approach the upper bound, DiT-MoS introduces three strategies including diverse training data splitting to increase the classifiers' diversity, adversarial selector-classifiers training to ensure synergistic interactions thereby maximizing their complementarity, and heterogeneous feature aggregation to improve the capacity of classifiers. We further propose a network slicing technique to alleviate the extra memory overhead incurred by feature aggregation. We deploy DiTMoS on the Neucleo STM32F767ZI board and evaluate it based on three time-series datasets for human activity recognition, keywords spotting, and emotion recognition, respectively. The experiment results manifest that: (a) DiTMoS achieves up to 13.4% accuracy improvement compared to the best baseline; (b) network slicing almost completely eliminates the memory overhead incurred by feature aggregation with a marginal increase of latency. Code is released at https//github.com/TheMaXiao/DiTMoS

I. Introduction

Internet of Things (IoTs), equipped with diverse miniatur-ized and low-power sensors, have catalyzed a remarkable surge in continuous and cost -effective pervasive sensing applications in recent years, encompassing environmental monitoring [1], asset tracking [2], and on-body human sensing [3]. On the other hand, advancements in artificial intelligence, especially deep neural networks (DNNs), enable effective extraction of both the explicit and implicit information within the substantial volume of sensor data, leading to enhanced sensing performance and more precise comprehension of the context [4]. For example, recent transformer DNNs like SwinV2 [5] can boost the image recognition accuracy up to 90.17% compared to 52.9% achievable by conventional machining learning like support vector machine (SVM) on the ImageNet dataset [6]. Figure 1.

Illustration of (a) top-down and (b) bottom-up methodology.