SensiX: A System for Best-Effort Inference of Machine Learning Models in Multi-Device Environments

Multiple sensory devices on and around us are on the rise and require us to redesign a system to make an inference of ML models accurate, robust, and efficient at the deployment time. While this multiplicity opens up an exciting opportunity to leverage sensor redundancy and high availability, it is still extremely challenging to benefit from such multiplicity and boost the runtime performance of deployed ML models without requiring model retraining and engineering. From our experience of deploying ML models in multi-device environments, we uncovered two prime caveats, device and data variabilities that affect the runtime performance of ML models. To this end, we develop an ML system that addresses these variabilities without modifying deployed models by building on prior algorithmic work. It decouples model execution from sensor data and employs two essential operations between them: a) device-to-device data translation for principled mapping of training and inference data and b) quality-aware dynamic selection for systematically choosing the execution pipeline as a function of runtime accuracy. We develop and evaluate a prototype system on wearable devices with motion and audio-based models. The experimental results show that ML models achieve a 7-13% increase in runtime accuracy solely by running on top of our system, and the increase goes up to 30% in dynamic environments. This performance gain comes at the expense of 3 mW on the host device.

1 Introduction

Sensory connected devices are now pervasive. Mobile, wearable, and IoT devices on and around us are increasingly embracing bleeding-edge machine learning (ML) models to uncover remarkable sensory applications [1], [2], [3], [4], [5]. In this transformation, we are observing the emergence of multi-device systems as a natural course of multiple sensory devices surrounding us. A concrete example of this is manifested in personal devices/wearables and IoT devices deployed in our homes. Studies predict more than 9 devices per person by the year 2025 [6] with diverse sensing capabilities (e.g., motion, vision, acoustics, RF). Therefore, we believe that focusing on efficient and accurate sensing in a multi-device environment is of paramount importance moving forward.