RPLNet: Object-Object Affordance Recognition via Relational Phrase Learning

Object-object affordance recognition aims to recognize the interactive relations between an object and other objects, which plays a crucial role in task decision-making and object selection of industrial robots. To address the problem of interference from complex interaction relations, we propose to recognize object-object affordances via relational phrase learning. The relational phrases are used as knowledge prior to improve the affordance expression. In addition, we propose a multi-scale feature pooling and aggregation module to enhance the visual feature representation of images. We redesign the initial block of transformer decoder to model the visual and phrase semantic features, improving the recognition performance. We collect and annotate object-object interaction images from the robot’s view, and train and test our model on them. The experimental results show that our approach achieves the best performance compared to the state-of-the-art methods.