Immersive Assistance System for Intuitive Robot Programming using Mixed-Reality and Digital Twin

Enabling technologies influence our everyday life. Robot manipulators are prominent examples in the industrial realm. They carry out repetitive production tasks with repeatable accuracy at high speed for a wide range of payloads. However, enterprises still refrain from adopting industrial robots. Reasons include the complexity and efforts necessary to program and adapt the robot behavior. In fact, employees are seldom suitably skilled to complete this goal and hiring experts increases production costs. As customer demands fluctuate in terms of volume and individualization, this situation gets worse since time windows to reprogram the robot might be substantially shortened and standard vendor-specific motion patterns fail to accommodate irregular Cartesian profiles. To address these challenges, this work combines mixed-reality and digital twin technologies to develop a natural, dynamic, and fast approach to (re-)program robots. It decouples motion planning driven by augmented reality, application-agnostic connectivity, and physical motion execution to augment even novice users with the capability to move a robot between freely and intuitively specified Cartesian frames without writing any line of code. The usefulness and effectiveness of the approach is demonstrated in many illustrations in practice.