Social Relationship Recognition Based on Shifted Windows Transformer and Dynamic Attention Graph Neural Network

Image-based social relationship recognition is a challenging computer vision task. Understanding social relationships in images helps to build better intelligent systems. The features of people or objects extracted from the image will greatly affect the recognition of social relationships. In addition, most previous studies are limited to independently inferring the relationships between people. That is, if there are three pairs of relationships in an image, the network will process them three times independently rather than jointly. In order to get more detailed features and avoid unreasonable social relationships caused by inferring social relationships independently, in this paper, we propose to use Shifted Windows Transformer (Swin Transformer) to extract more expressive features and graph neural network based on dynamic attention to jointly infer all relationships between people in images. Experimental results show that the proposed method improves the performance of social relationship recognition on two benchmark datasets.