Higher Order Polynomial Transformer for Fine-Grained Freezing of Gait Detection

Freezing of Gait (FoG) is a common symptom of Parkinson’s disease (PD), manifesting as a brief, episodic absence, or marked reduction in walking, despite a patient’s intention to move. Clinical assessment of FoG events from manual observations by experts is both time-consuming and highly subjective. Therefore, machine learning-based FoG identification methods would be desirable. In this article, we address this task as a fine-grained human action recognition problem based on vision inputs. A novel deep learning architecture, namely, higher order polynomial transformer (HP-Transformer), is proposed to incorporate pose and appearance feature sequences to formulate fine-grained FoG patterns. In particular, a higher order self-attention mechanism is proposed based on higher order polynomials. To this end, linear, bilinear, and trilinear transformers are formulated in pursuit of discriminative fine-grained representations. These representations are treated as multiple streams and further fused by a cross-order fusion strategy for FoG detection. Comprehensive experiments on a large in-house dataset collected during clinical assessments demonstrate the effectiveness of the proposed method, and an area under the receiver operating characteristic (ROC) curve (AUC) of 0.92 is achieved for detecting FoG.

I. Introduction

Parkinson’s disease (PD) is a chronic neurodegenerative disorder, which progressively damages the central nervous system of the patients [1]. Freezing of Gait (FoG) is regarded as one of the most debilitating and least understood symptoms in PD [2]. It manifests as a sudden and brief episode when the feet of a patient get stuck during walking, despite the intention to walk [3], [4]. As the disease progresses, FoG happens more frequently and severely. This poses a risk for falls and affects mobility, independence, and PD patients’ quality of life [5]. Therefore, effective identification and quantification of FoG events are crucial for PD assessment and management [6].