Face Tracking Strategy Based on Manipulability of a 7-DOF Robot Arm and Head Motion Intention Ellipsoids

Nurses recognize facial expressions or eye motions to monitor a patient’s condition in the intensive care unit (ICU), for example, pain, agitation, and delirium. However, there are no instruments that can record the facial expression or eye motion accurately like an ECG monitor. To tackle this issue, we develop a face tracking strategy using a 7-DOF robot arm with a camera mounted on the end-effector. First, we constrain the linear and angular velocities of head motion intention to ellipsoids which are determined by the patient’s head pose and the geometry of hospital beds, named head motion intention ellipsoids (HMIEs). Moreover, we defined manipulability ellipsoids (MEs) of the 7-DOF robot arm based on Jacobian matrix, which is adjusted in the null space during the tracking. We calculate the optimal configuration of the camera with the feedback of the head configuration while minimizing the difference between HMIEs and MEs. Simulation experimental results verified that the proposed face tracking strategy outperforms the visual servoing control strategy only based on the pseudo-inverse of the Jacobian matrix.