VRCK: A Virtual Reality Teaching System based on Generation and Editing of Constrained Keyframes

Virtual reality (VR) demonstration is safe, convenient, low-cost, and has easy access to learning data. Keyframes have been extensively used in learning from demonstration (LfD) to teach robots skills, especially when the skills need to be further updated and adapted. However, few works use keyframe-based methods in VR demonstrations to combine the advantages of both. Besides, human annotations and multiple demonstrations are always required to generate keyframe models, which causes problems of large demonstration workload, poor interpretability, and low generalization performance. To tackle the issues mentioned above, we propose VRCK: a VR teaching system based on generation and editing of constrained keyframes. In this system, only one demonstration is needed to generate the skill keyframes, path keyframes and additional keyframes. Constraints can be generated automatically using object detection and binocular localization algorithms. During the VR teaching process, users can visualize keyframes and constraints, and use UI to bind and edit them at any time. Finally, three task cases and a physical experiment are provided to validate our system and illustrate how it enables intratask generalization.