6D-Pose Estimation for Manipulation in Retail Robotics using the Inference-embedded OAK-D Camera

The socio-economic need for service robots has become more evident during the ongoing pandemic. To boost their deployment, robots need to improve their manipulation capabilities, which includes solving one of the biggest challenges: determine the position and orientation of the target objects. While conventional approaches use markers which require constant maintenance, deep-learning-based approaches require a host computer with high specifications. In this paper, we propose a 6D-pose estimation system whose segmentation algorithm is embedded into OAK-D, a camera capable of running neural networks on-board, which reduces the host requirements. Furthermore, we propose a point cloud selection method to increase the accuracy of the 6D-pose estimation. We test our solution in a convenience store setup where we mount the OAK-D camera on a mobile robot developed for straightening and disposing of items, and whose manipulation success depends on 6D-pose estimation. We evaluate the accuracy of our solution by comparing the estimated 6D-pose of eight items to the ground truth. Finally, we discuss technical challenges faced during the integration of the proposed solution into a fully autonomous robot.