Temporal–Spatial Dynamic Convolutional Neural Network for Human Activity Recognition Using Wearable Sensors

The sensor-based human activity recognition (SHAR) task seeks to recognize signals collected by various sensors embedded in intelligent devices to assist people in their daily lives. Motivated by the success of deep learning, many researchers are studying combining deep learning with SHAR. The key to implementing SHAR with deep learning lies in facilitating model performance and maintaining efficiency when the model is performed on resource-constrained devices. To address this challenge, we present an effective sensor signal representation method, termed the temporal–spatial dynamic convolutional network, to recognize human activity. Temporal–spatial dynamic convolution (TS-DyConv) aims to dynamically learn the convolutional kernels weighted with attention generated along the temporal and spatial kernel spaces. In this way, the TS-DyConv can diversify the kernel to enhance the sensor signal’s recognition capabilities without raising complexity and maintaining efficiency. Extensive experiments conducted on three benchmark SHAR datasets, e.g., OPPORTUNITY, PAMAP2, and USC-HAD, demonstrate the superiority of our method over the deep learning baselines and existing SHAR works.

I. Introduction

Recent years have witnessed the growing popularity of intelligent devices, including smartphones, smartwatches, and other smart wearable devices. The expansion of these devices encourages the massive production of raw signal, facilitating research to improve people’s quality of life. With the development of technology, sensor-based human activity recognition (SHAR), one of the most vital research areas, has been utilized in many application fields, such as healthcare [1], fitness [2], and workout monitoring [3].