A Real-time Hand Motion Detection System for Unsupervised Home Training

Hand motion tracking plays a vital role in health-care like Physical therapy (PT) rehabilitation that helps the patients restore their physical movement of the hand. These treatments are taken by patients suffering from stroke, accidents, and any other kind of neurological disorder. We have developed a low-cost system to track hand movements and detect the gestures of hand for unsupervised home training. The system integrated a convolutional neural network-based hand motion system and a gesture detection system to serve a training session sequential for hand movement rehabilitation. We combined part of open datasets and our novel dataset(total: 16605, 4 labels) for the final training, six directions, and four gestures were predicted in real-time based on our proposed model. An adaptive GUI application was developed to respond to individual performance patterns. This system can be deployed easily on a laptop/PC with a web camera, which makes it easier and low-costs for a doctor to track the hand movement of the patients and also support to quantify the improvements index after several training sessions. These tracked data can also be stored and sent to the remote clinic center and used for further studies. This kind of system would be beneficial to the patients and the training center intelligently. It could be deployed at home or the clinic with more optimization of functions such as safety measures.