Robot Skill Learning and Generalization based on Human-robot Collaboration and Computer Vision

The paper presents a novel approach for human-robot skill transferring. Firstly, we propose a method that combines dynamic time warping (DTW) with the Gaussian mixture model (GMM) to reconstruct the demonstrated skills, including reference path and pose, for each workpiece. Secondly, we integrate hand-eye coordination to facilitate dataset creation and employ YOLOv8 for model training. Finally, we utilize a neural network to obtain the current workpiece's category and pose information and then transform the previous skills into the current workpiece coordinate system. The effectiveness and robustness of our proposed method have been validated on a 7-DOF Sawyer robot equipped with a camera.