SKGHOI: Spatial-Semantic Knowledge Graph for Human-Object Interaction Detection

Human-Object Interaction (HOI) detection is a challenging computer vision task that aims to recognize and understand the interactions between humans and objects in images or videos. Existing techniques heavily rely on appearance-based features and computationally expensive transformer models for semantic representation. In this paper, we propose SKGHOI (Spatial-Semantic Knowledge Graph for HOI), a novel graph-based approach that efficiently captures semantic representation by integrating spatial and semantic knowledge. SKGHOI constructs a graph with interaction components as nodes and spatial relationships as edges. Our approach leverages spatial and semantic encoders to extract spatial and semantic information, which are then fused to create a knowledge graph that captures the semantic representation of HOIs. Compared to existing methods, SKGHOI offers computational efficiency and the incorporation of prior knowledge, making it practical for real-world applications. Experimental evaluations on the widely-used HICO-DET datasets demonstrate that SKGHOI outperforms state-of-the-art graph-based methods by a significant margin, showcasing its effectiveness and potential for improving the accuracy and efficiency of HOI detection.