WatchLogger: Keyboard Typing Words Recognition Based on Smartwatch | IEEE Conference Publication | IEEE Xplore

WatchLogger: Keyboard Typing Words Recognition Based on Smartwatch


Abstract:

Nowadays more and more people are wearing smart-watches in their daily lives. The various sensors embedded in smartwatches bring the ability to evaluate users' status as ...Show More

Abstract:

Nowadays more and more people are wearing smart-watches in their daily lives. The various sensors embedded in smartwatches bring the ability to evaluate users' status as well as the risk of privacy issues. For example, if users are typing on key-boards while wearing smartwatches, the attacker could know the typing contents from the sensor data collected by the malicious applications that are installed on the targets' smartwatches. In this paper, we propose WatchLogger, the framework using audio and accelerometer signals to recognize the English words being typed, for demonstrating how to implement the smartwatch-based side-channel attack. Different from the previous studies that focused on the recognition of each key or pair of keys being pressed, WatchLogger aims to perform recognition on the scale of words. To achieve this goal, WatchLogger exploits the audio signals for segmentation and the accelerometer signals for classification. In addition, we propose an ensemble classification model to deal with the problem caused by too many words. At last, we build the dataset WTW -100 with 100 classes of words and 100 samples for each class, and we conduct experiments on the dataset. The experimental results show an accuracy of 98.5 % for keystroke recognition and 91.5 % for word classification, showing a considerable performance of WatchLogger.
Date of Conference: 29 November 2023 - 01 December 2023
Date Added to IEEE Xplore: 30 January 2024
ISBN Information:
Conference Location: Kyoto, Japan

I. Introduction

Smartwatches are becoming ubiquitous in people's daily lives. There are various sensors embedded in the smartwatch, e.g., the optical sensor, accelerometer, and microphone, making it possible to provide useful applications for evaluating users' status, such as heart rate monitoring, sports logging, and fall detection. However, these sensors also carry sensitive information that could be potentially unsafe, especially when the devices are compact and non-intrusive, users could easily neglect them and relax their vigilance. The attacker can collect sensor data from the target users through the malicious application installed on their smartwatch and transmit the data to the attacker's machine so that the attacker can infer certain information about the targets, posing a threat to the privacy issues of smartwatch users.

References

References is not available for this document.