Mixed Reality Teleoperation Assistance for Direct Control of Humanoids

Teleoperation plays a crucial role in enabling robot operations in challenging environments, yet existing limitations in effectiveness and accuracy necessitate the development of innovative strategies for improving teleoperated tasks. This letter introduces a novel approach that utilizes mixed reality and assistive autonomy to enhance the efficiency and precision of humanoid robot teleoperation. By leveraging Probabilistic Movement Primitives, object detection, and Affordance Templates, the assistance combines user motion with autonomous capabilities, achieving task efficiency while maintaining human-like robot motion. Experiments and feasibility studies on the Nadia robot confirm the effectiveness of the proposed framework.

I. Introduction

Several of the most compelling instances of humanoids performing valuable tasks have been through teleoperation [1]. However, despite years of research in this domain, teleoperated humanoid robots continue to show substantial limitations when executing tasks under user control. Traditional direct control of robots through the user's own motions offers an intuitive approach for interacting with the remote environment [1], [2], [3]. However, differences in kinematics between humans and robots can lead to errors and multiple attempts for task completion, making this approach not very effective and time-consuming, and even lead to operator frustration.