Tactile Object Recognition Using Fluid-Type Sensor and Deep Learning

Tactile data are essential in object perception. Based on such data, various objects can be recognized and differentiated. The fluid-type tactile sensor employs a colored fluid captured between an elastic skin and a transparent plate where the color intensity of the fluid observed by a camera is directly related to the deformation of the skin. In this letter, the fluid-type sensor is restructured by employing an optimization method that is sensitive to skin deformation. Moreover, the implementation of nonopaque skin is proposed to reveal surface color features in addition to the shape of the object in tactile images. A convolutional neural network (CNN), a neural network based on transfer learning and dropout layer was implemented to classify objects based on their tactile images, which is a combination of shape of the objects and their surface color features. Although the number of training samples is small, an accuracy of 94.2% is achieved. The proposed sensor is 3-D printable and can be fabricated at a low cost.