Attitude Recognition of Grasped Objects by Pneumatic Three-Jaw Manipulator with Flexible Electronic Skin

When our hand grasping an object, the haptic system in the skin can perceive the pose of the object with high accuracy, even if the visual system is absent. In this work, we attach an electronic skin (E-skin) with an array of piezoresistive sensors to a pneumatic three-jaw manipulator to recognize the pose of the object grasped by the manipulator. With a 16×16 matrix pressure signal acquisition system, we collected about 260,000 sets of pressure data with spherical coordinate pose labels, trained the convolutional neural network (CNN) model and validated it with about 60,000 sets of data. The results showed that our system can recognize the attitude of the grasped objects with minimum errors of azimuth angle θ and polar angle ϕ in the pose angle reached 6.011° and 2.464°, respectively. For objects of different shapes, we employed transfer learning in the CNN model and the system can successfully perceive the postures of them with low errors.