TIODT: Touch-free Intuitive Operation Digital Twin Platform for Minimally Invasive Surgical Robot

Precise and intuitive control interfaces are vital in the field of surgical robotics to enhance surgical outcomes. This study introduces a novel approach to touch-free control of a surgical robot using a digital twin framework. The system leverages advanced hand tracking, gesture recognition, and real-time gesture mapping techniques to enable accurate and intuitive control. The comprehensive visual feedback enhances the surgeon’s understanding and decision-making during procedures. Detailed characterization and capture experiments were conducted to verify the proposed system and the results show that the digital twin platform allows surgeons to visualize the surgical site from different perspectives and overlay additional information for improved situational awareness.