Natural Robot Guidance using Transformers

Recently unveiled intelligent industrial robots are endowed with force-sensitive capabilities. Beyond enhanced safety in physical interactions with humans and the environment, the adaptation to unpredicted situations as well as consistent and flexible robotized manipulations during servicing and manufacturing are benefits of the compliance control of these robots. Despite these advantages, the usage of compliant robots is inhibited by the lack of adequately skilled collaborators when it comes to program and customize force-sensitive motion behaviors of robots. Furthermore, the adoption of approaches that rely upon machine learning/artificial intelligence (ML/AI) to automate the customization of a soft or stiff robot behavior faces a lengthy roll out of projects and settings, scarce and quality-limited data, as well as high energy consumption to train multiple ML/AI models that support the plethora of tasks the robot might be involved in. To address time-, cost-, and quality-related issues induced by these challenges and make compliance programming accessible to even novices, this work in progress aims at taking advantage of natural language instructions to customize compliant robot behaviors. Human intentions inferred from instructions are associated with desired levels of force-sensitiveness, objects near the robot, and combined with an admittance model to generate a compliant deformation of planned robot trajectories. This work aims at leveraging pre-trained foundation models to cope with data limitation and (self) attention of transformers to capture correlations, contexts, and autoregressively yield compliant motions, thus naturally simplifying compliance settings for robot guidance.