Human-Robot Collaboration Through a Multi-Scale Graph Convolution Neural Network With Temporal Attention

Collaborative robots sensing and understanding the movements and intentions of their human partners are crucial for realizing human-robot collaboration. Human skeleton sequences are widely recognized as a kind of data with great application potential in human action recognition. In this letter, a multi-scale skeleton-based human action recognition network is proposed, which leverages a spatio-temporal attention mechanism. The network achieves high-accuracy human action prediction by aggregating multi-level key point features of the skeleton and applying the spatio-temporal attention mechanism to extract key temporal information features. In addition, a human action skeleton dataset containing eight different categories is collected for a human-robot collaboration task, where the human activity recognition network predicts skeleton sequences from a camera and the collaborating robot makes collaborative actions based on the predicted actions. In this study, the performance of the proposed method is compared with state-of-the-art human action recognition methods and ablation experiments are performed. The results show that the multi-scale spatio-temporal graph convolutional neural network has an action recognition accuracy of 94.16%. The effectiveness of the method is also verified by performing human-robot collaboration experiments on a real robot platform in a laboratory environment.

I. Introduction

Robotics advancements have made it possible for machines to function with ease in challenging environments, effectively addressing the issue of a labor shortage. However, robots still rely heavily on pre-programming to perform specific tasks, which can limit their flexibility and ability to perform complex actions that humans can easily perform. As the demand for robots to perform delicate and flexible tasks in complex environments continues to grow, collaborative robots are needed to bridge the gap in human-robot interaction. Collaborative robots can help overcome the limitations of pre-programming by working alongside humans and using their cognitive abilities to adapt to changing situations and perform tasks that are difficult for robots to perform on their own.