YOLOv7-based Visual Servoing on 2-DOF Manipulator Robot

Manipulator robot are widely used in industry to handle repetitive tasks such as moving components. To compensate for conventional robot control weakness, visual servoing control technique was developed. Feature extraction in visual servoing must achieve fast response with precise accuracy. Current research in deep learning based object detection algorithm provides satisfactory result and able to detect objects in real time. YOLOv7 set the new state-of-the-art performance, with better accuracy and speed compared to other object detectors. In this work we compare various models of YOLOv7 and YOLOv5 in accuracy and speed. The results shows that YOLOv7tiny achieved 12.3 FPS with Mean-Squared Error of pixel coordinate (u,v) consecutively 0.0092 pixel/width and 0.00739 pixel/height when applied to edge device. We then designed and implemented YOLOv7tiny-based Image Based Visual Servoing (IBVS) in 2 joints of PUMA260 robot Our implementation successfully tracked the designated object, making the center point of the object corresponds to our desired setpoint (160, 100) on less than 30 visual subsystem iterations. This shows than YOLOv7 can be used for manipulator visual servoing applications, and further research can develop 6-DOF (Degree of Freedom) visual servoing control using interaction matrices.