Cross-Modal Hashing for Material Surface Properties Fusion

In the tasks of robotic interaction with its surrounding environments, it is critical to judge the properties of the surrounding objects quickly and accurately. Existing high-definition cameras and tactile sensors can collect high-precision visual and tactile information, but it cannot comprehensively describe the surface properties of objects from single-modality. Hence, how to establish the cross-modal fusion between visual and tactile plays an important role in material surface judgement. To achieve above target, we propose a Hashing Learning For Cross-Modal Material Surface Retrieval. First, we apply hashing learning to cross-modal fusion, which can reduce storage costs and improve query efficiency. Then, a self-attention network is proposed to eliminate redundant information, in other words, to extract cross-modal correlation information between different modalities. Finally, in the process of generating hashing codes, a center constraint is applied to protect intra-modal structural information. Experimental results on the publicly available data set demonstrate the effectiveness of the proposed method.