Research on Multimodal Fusion Recognition Method of Upper Limb Motion Patterns

In order to solve the problems of single movement pattern recognition information and low recognition accuracy of multijoint upper limb exoskeleton rehabilitation training, a multimodal information fusion method with human surface electromyography (sEMG) and electrocardiogram (ECG) was proposed, and an Inception-Sim model for upper limb motion pattern recognition was designed. Integrating the advantages of multimodal information, inspired by the convolutional neural network processing image classification problem, the original signal was converted into a Gramian angular summation/difference fields-histogram of oriented gradient (GASF/GADF-HOG) image based on the principle of Grameen angle superposition/difference field, and the directional gradient histogram feature of the GASF/GADF image was extracted. The Inception-Sim model was constructed based on the Inception V3 model, and the human motion pattern recognition was completed on the basis of the transfer learning network. VGG16, ResNet-50, and other backbone networks were selected as comparison models. The recognition accuracy of each motion pattern for all participants reaches up to 90%, which is better than that of the control model. The average iteration speed of the proposed Inception-Sim model improved by about 21% compared to the control model. The experimental results show that the proposed multimodal information fusion recognition method can improve the accuracy and iteration speed of the upper limb motion recognition mode and then improve the effect of upper limb rehabilitation training.