Towards Deep Learning-Based Classification of Multiple Gestures using Sonomyography for Prosthetic Control Applications

Intuitive control of multi articulated prosthetic devices remains a challenge due to inherent shortcomings in surface electromyography-based control modalities. As a result, sonomyography or ultrasound imaging-based methods of muscle activity sensing and control of bionics devices have emerged as an attractive alternative to traditional myoelectric control methodologies. However, most sonomyography-based techniques involve extraction and analysis of image features from B-mode ultrasound images to predict gestures. Additionally, ultrasound probe shift during the experiment has a detrimental effect on prediction accuracies. Therefore, to address these challenges we develop a deep-learning-based method to automatically identify four hand gestures: hand open, power grasp, index pointing, and tripod. Hand orientation variations were included by performing lateral and transverse hand movement, wrist supination, and pronation. We fine-tuned four deep-learning models (AlexNet, ResNet18, DarkNet19, and VGG16) using 4000 2D ultrasound images of forearm muscle deformation. Results show that a fine-tuned VGG16 outperformed other architectures by achieving 98.7 % cross-validation accuracy.