Open Set User Identification Using Gait Pattern Analysis Based on Ensemble Deep Neural Network

The human gait is a unique biological characteristic of human beings and can be used for recognition task. Collecting gait information using wearable devices and identifying people with this data has been widely researched in recent times. Though many gait recognition studies have been conducted, few studies address the open set recognition problem where an unknown sample may be input into the system during the testing phase. A working model that overcomes this problem should correctly recognize whether a sample has never been seen before during training while maintaining high classification accuracy of known samples. We address the open set gait recognition problem by proposing a system that maps multi-modal unit step data gathered from insoles to an embedding vector in a latent space. Specifically, we collected the the time-series using sensor equipped insoles, and we process the raw multi-modal time-series into unit fragments by slicing them so that they can be used as input for an ensemble network made up of a convolutional neural network, recurrent neural network, and self-attention network. The proposed ensemble network is trained with the additive angular margin loss. The resulting embedding vector is used to recognize which subject the unit step belongs to through a decision function derived using a one-class support vector machine that requires only a few unit steps per subject for few-shot training. This decision function also determines whether a unit step does not belong to any of the subjects used to train the one-class support vector machine. We show that our model maintains high classification accuracy for known unit step subjects while correctly recognizing which unit steps were never used in the training phase. We demonstrate the performance of our proposed system in an experimental study using multi-modal sensing data.