Fault-Tolerant and Adaptive Visual Servoing for Capturing Moving Objects

This article focuses on an adaptive and fault-tolerant vision-guided robotic system that enables to choose the most appropriate control action if partial or complete failure of the vision system in the short term occurs. Moreover, the autonomous robotic system takes physical and operational constraints into account to perform the demands of a specific visual servoing task in a way to minimize a cost function. A hierarchical control architecture is developed based on interwoven integration of a variant of the iterative closest point image registration, a constrained noise-adaptive Kalman filter, a fault detection logic and recovery system, together with a constrained optimal path planner. The dynamic estimator estimates unknown states and uncertain parameters required for motion prediction while imposing a set of inequality constraints for consistency of the estimation process and adjusting adaptively the Kalman filter parameters in the face of unexpected vision errors. It is followed by the implementation of a fault recovery strategy based on a fault detection logic that monitors the health of the visual feedback using the metric fit error of the image registration. Subsequently, the estimated/predicted pose and parameters are passed to an optimal path planner in order to bring the robot end-effector to the grasping point of a moving target as quickly as possible subject to multiple constraints, such as acceleration limit, smooth capture, and line-of-sight angle of the target. Experimental results demonstrated such a visual servoing system succeeded to capture a free-floating object despite the complete failure of the vision system due to occlusion in the last 10 s prior to approach and capture operation.