Human Activity Recognition Using Spectrograms of Binary Motion Sensor Data

Human activity recognition is at the basis of several applications in the smart living domain, such as energy management, elder care, and health management. Human activity recognition research can be divided into two categories, depending on the type of sensors used: wearable sensors, such as those found in mobile phones and smart watches, and ambient sensors, such as motion sensors or cameras placed in the environment. Among ambient sensors, binary sensors are often perceived as less invasive than sensors that collect video, audio, or biometric data. However, the performance of classifiers trained on binary sensor data is often lower since the data inherently contains less information. In this paper, we propose a non-intrusive human activity recognition framework that only exploits binary sensor data and results in high classification accuracy. Our approach is inspired by audio and image processing applied to binary sensors. Specifically, we exploit the Short-Time Fourier Transform (STFT) to extract features from binary data. These features are used to train a hybrid machine learning model which pairs Convolutional Neural Network (CNN) with a Long-Short-Term Memory (LTSM) architecture. We use a real dataset of human activities monitored through binary sensor data for evaluating the impact of the features on classifier performance. Results show that the proposed method significantly outperforms state-of-the-art solutions, requiring minimal training data needed to achieve a given level of accuracy.