Patient performance evaluation using Kinect and Monte Carlo-based finger tracking

The growing use of Virtual Reality (VR) in rehabilitation is justified by a number of advantages, such as an increase of patient motivation, repetitiveness of learning trials, possibility to tailor treatment to individual subject, safety of the environment, quantitative patient improvement assessment, and remote data access. This paper proposes a novel low-cost evaluation method of patient performance in task-oriented hand rehabilitation grounded on two key elements: a Virtual Environment (VE) which the patient has to interact with, and the Microsoft Kinect motion sensing device, which is used to fully interact with the VE and to feed back patient movements in order to perform an off-line analysis. To this purpose, the VE is equipped with a virtual hand and virtual objects the patient has to interact with. In order to make the interaction between patient and VE possible, a robust marker-based finger tracking algorithm has been developed by using Bayesian estimation methods. In the proposed framework, the hand movements involved in daily activities are performed off-line by the therapist and are tracked by using the Kinect camera. The estimated hand joint trajectories are provided in input to a virtual hand model developed with the Matlab Virtual Reality Toolbox. The virtual hand reproduces the movements performed by the therapist and the patient is asked to imitate them. User motor improvements can be monitored by the Kinect camera, superimposing the therapist finger trajectories on the patient finger trajectories. The error between the two trajectories can be used for evaluating the patient residual mobility. The proposed system can be easily applied to home-based rehabilitation.