Dynamic Inference via Localizing Semantic Intervals in Sensor Data for Budget-Tunable Activity Recognition

During recent years, deep convolutional neural networks have demonstrated dominant performance in human activity recognition (HAR) using wearable sensors. However, they often come at high computational cost when fueled with fixed-length sliding window. This article primarily aims to accelerate activity inference from a novel perspective of reducing temporal redundancy in sensor data. Inspired by the fact that not all time intervals within a window are activity-relevant, we formulate the activity prediction problem as a dynamic inference process by continuously attending to a sequence of small activity-discriminative intervals, which are selected from an original window by progressively predicting the discriminative importance of each interval with an interpretable interval proposal network. The dynamic process can adaptively decide when to halt for each individual sample, which considerably avoids excessive computation by letting “easy” activity exit as early as possible while progressively focusing on small salient intervals for “hard” activity. Given a limited budget, the accuracy-cost tradeoff can be flexibly and precisely controlled via tuning confidence thresholds online without requiring to be retrained from scratch—a practical requirement in real-world HAR applications. Extensive experiments on several standard benchmarks including University of California-Irvine-Human Activity Recognition (UCI-HAR), wireless sensor data mining (WISDM), University of Southern California-Human Activity Dataset (USC-HAD), and Weakly Labeled dataset demonstrate that our dynamic inference process significantly outperforms previous static methods according to theoretical and practical computational efficiency.