Development of a Platform for Novel Intuitive Control of Robotic Manipulators using Augmented Reality and Cartesian Force Control

This paper presents a novel approach to intuitive force-controlled motion planning on collaborative robots using augmented reality (AR) technology. A user-friendly interface is developed that grants the operator complete control over the robot manipulator. By employing a mixed reality head-mounted display (HMD) as the interface, virtual content is overlaid, enabling the operator to interact seamlessly with the robotic system. The interface provides extensive data on the robot’s status, including joint position, velocity, and the applied force on the robot’s flange. Operators can issue motion commands in both joint and Cartesian space, intuitively plan robot paths using waypoints, and execute force-controlled motion by defining control points around an object. Visual feedback in the form of superimposed sliders indicates the force to be exerted by the robot on the object. These sliders allow dynamic and intuitive adjustment of forces in Cartesian space, minimizing the need for extensive programming. Safety is a primary concern, and to address it, a virtual model of the robot is superimposed on the work environment, providing a preview of the motion. This preview displays the current and final positions of each joint before execution. The human-robot interface and virtual content are built using the Unity3D game engine, while reliable data transmission and processing between the HMD and the robot controller are facilitated by the Robot Operating System (ROS). This approach offers an intuitive and safer method for controlling collaborative robots, empowering operators with greater precision and ease in robot motions. The proposed approach has significant potential to streamline robot programming, enhance efficiency, and improve safety across a wide range of applications involving collaborative robots.