Performing manufacturing tasks with a mobile manipulator: from motion planning to sensor based motion control

We present a framework combining mobile manipulation planning and sensor based motion control. The result of manipulation planning is a reference robot trajectory composed of segments. For each segment, a hierarchical task based controller is automatically built. Some of the segments correspond to motions where some parts of the robot and of the environment are known to be close to each other. The corresponding controller implements a task of relative pose between these elements. The task error is measured by vision sensors using AprilTags. The higher priority of this task with respect to the reference joint trajectory greatly improves the accuracy of the task affected by the poor accuracy of the robot base localization and other sources of error.