AvTF: A visual-tactile fingertip with active sensing and manipulation

In the interaction between dexterous hands and the environment, perception and operation are inseparable. Perception provides information and feedback for operations, and operations in turn affect the perception. Current VTS(visual-tactile sensors) lack initiative and cannot sense the state of the operated object in the world coordinate system. The AvTF(Active visual-tactile Fingertip) introduced in this article is a vision-based fingertip (finger-shaped full-field sensing) tactile sensor that uses a camera to capture tactile images. Sensing and operating are integrated by adding active DoF (Degree of Freedom). Camera modules under different coordinate systems can provide various perspectives for sensors. Through a series of experimental verifications, AvTF has achieved real-time feedback on the grasping and manipulation of objects in the hand, providing a new solution for dexterous hands to fuse sensing and operation better.

I. INTRODUCTION

Sensors can help a robot sense and understand its surroundings, allowing it to work in complex and dynamic environments. The VTS is an optical sensor that provides the manipulator with more comprehensive sensing capabilities by fusing visual and tactile information. At present, VTS can obtain tactile information such as hardness, texture, and deformation of the object surface[1], [2], [3]. This fusion allows the robot to more accurately perceive objects in the environment, thereby improving its adaptability to complex tasks.