Redundancy-Based Visual Tool Center Point Pose Estimation for Long-Reach Manipulators

In this paper, we study a visual sensing scheme for 6 degree-of-freedom (DOF) tool center point (TCP) pose estimation of large-scale, long-reach manipulators. A sensor system is proposed, designed especially for mining manipulators, comprising a stereo camera running a simultaneous localization and mapping (SLAM) algorithm near the TCP and multiple cameras that track a fiducial marker attached near the stereo camera. In essence, the TCP pose is formulated using two different routes in a co-operative (eye-in-hand/eye-to-hand) manner using data fusion, with the goal of increasing the system's fault tolerance and robustness via sensor redundancy. The system is studied in offline data analysis based on real-world measurements recorded using a hydraulic 6 DOF robotic manipulator with a 5 m reach. The SLAM pose trajectory is obtained using the open source ORB-SLAM2 Stereo algorithm, whereas marker-based tracking is realized with a high-end motion capture system. For reference measurements, the pose trajectory is also formulated using joint encoders and a kinematic model of the manipulator. Results of the 6 DOF pose estimation using the proposed sensor system are presented, with future work and key challenges also highlighted.