A 3D Camera Protocol for Object Pose Estimation from Point Cloud in Robot Operations

The utilization of industrial robots and cameras in industrial manufacturing has become widespread. However, the high cost of industrial 3D cameras, in particular, poses a significant challenge in the selection process. The advancements in RGB-D camera technology, which is significantly involved in the development of robotics and more affordable than industrial 3D cameras, are noteworthy. In this research, the Intel Realsense L515 RGB-D camera and the NVIDIA Jetson Xavier NX single board computer were selected for the implementation of an object pose estimation application. The object segmentation algorithm, YOLOv7, was proposed for object detection, which enables the calculation of the X, Y, and Z position of the object. Subsequently, a master and object segmentation point cloud was generated, and a point cloud preprocessing and registration methodology was proposed to determine the R
x
, R
y
, and Rz angles of the object, utilizing the Open3D library. In addition, an industrial robot interfacing via TCP/IP and serial communication to enable the transformation of the object pose into the robot pose for subsequent transmission is proposed. A web-based application was developed using the Django framework to facilitate RGB-D camera monitoring and parameter setting. The experiments were conducted using a three-way tube with a diameter of 25.4 mm. as the object, resulting in the RMS error in X, Y, Z, R
x
, R
y
, and Rz are 5.3, 4.3, 4.3 mm., 2.2, 1.4, 2.5 degrees respectively. The maximum error in X, Y, Z, R
x
, R
y
, and Rz are 15.7, 13.6, 6.5 mm., 4.9, 3.5, 7 degrees respectively.