FedAWR: An Interactive Federated Active Learning Framework for Air Writing Recognition

The rapid development of technology such as virtual reality and augmented reality, coupled with the reduced direct contact due to the COVID-19 pandemic, has led to the emergence of a more advanced mode of interaction: air handwriting. This new form of human-computer interaction allows users to input text by writing in the air freely. However, deploying and applying existing air handwriting recognition systems in real-world scenarios still presents challenges, particularly in real-time performance, privacy protection, and label scarcity. To address these challenges, we propose a federated active learning framework called FedAWR for air handwriting recognition tasks. FedAWR utilizes distributed learning to train a shared global model in the cloud from multiple user devices at the network's edge, while keeping the user's handwritten data local to ensure privacy. In addition, FedAWR employs an interactive active learning strategy to collect user-provided annotations for iterative training during the online federated learning process, bootstrapping personalized models for each client. To further enhance interactivity and real-time performance, we designed a lightweight recognition model, which is integrated into FedAWR. Finally, extensive experiments were conducted on real-world air handwritten datasets to validate the superiority of FedAWR.

I. Introduction

Recent advances in technology, such as virtual reality (VR) and augmented reality (AR), have enabled the creation of computer-generated perceptual information that transcends the boundary between physical and digital worlds. Handwriting, which was once limited to physical paper or digital surfaces, can now extend into the air. Air handwriting, a new form of contactless human-computer interaction, allows users to input text by writing freely in the air with their fingertips, using specific gestures to perform certain interactive operations.