Lobster detection using an Embedded 2D Vision System with a FANUC industrual robot

In this paper, two vision-based systems approaches are studied in order to guide the FANUC robot arm ‘FANUC LR Mate 200iD/7L’ to locate and manipulate lobsters. Different experiments are carried out using this robot in Robotics, Electronics and Industry 4.0 Laboratory of the Université de Moncton. The first approach aims to test the ability of the integrated vision system iRvision of the FANUC robot in detecting lobster body. The Geometric Pattern Matching (GPM) Locator and the Curved Surface Matching (CSM) Locator are studied and tested for this purpose. The second approach consists of a computer vision solution based on the YOLOv4 object detection algorithm, which was implemented and tested on the Nvidia Jetson NX embedded platform. Experimental results showed that, on the one hand, the iRVision system using GPM Locator has failed to detect lobster body parts. On the other hand, the CSM Locator has detected lobster body parts with a lower score detection and has exceeded 1 second for time detection. However, the embedded vision system based on the YOLOv4 model detected the main lobster body parts and achieved 99.29% for the main average precision (mAP) and 0.1806 second for detection time on the Nvidia jetson Xavier NX embedded platform.