Combining Pairwise Feature Matches from Device Trajectories for Biometric Authentication in Virtual Reality Environments

In this paper we provide an approach to perform seamless continual biometric authentication of users in virtual reality (VR) environments by combining position and orientation features from the headset, right hand controller, and left hand controller of a VR system. The rapid growth of VR in mission critical applications in military training, flight simulation, therapy, manufacturing, and education necessitates authentication of users based on their actions within the VR space as opposed to traditional PIN and password based approaches. To mimic goal-oriented interactions as they may occur in VR environments, we capture a VR dataset of trajectories from 33 users throwing a ball at a virtual target with 10 samples per user captured on a training day, and 10 samples on a test day. Due to the sparseness in the number of training samples per user, typical of realistic interactions, we perform authentication by using pairwise relationships between trajectories. Our approach uses a perceptron classifier to learn weights on the matches between position and orientation features on two trajectories from the headset and the hand controllers, such that a low classifier score is obtained for trajectories belonging to the same user, and a high score is obtained otherwise. We also perform extensive evaluation on the choice of position and orientation features, combination of devices, and choice of match metrics and trajectory alignment method on the accuracy, and demonstrate a maximum accuracy of 93.03% for matching 10 test actions per user by using orientation from the right hand controller and headset.