Vector Symbolic Scene Representation for Semantic Place Recognition

Most state-of-the-art methods do not explicitly use scene semantics for place recognition by the images. We address this problem and propose a new two-stage approach referred to as TSVLoc. It solves the place recognition task as the image retrieval problem and enriches any well-known method. In the first model-agnostic stage, any modern neural network model that does not directly use semantics, e.g., HF-Net, NetVLAD, or Patch-NetVLAD, can be used. In the second stage, we apply the Vector Symbolic Architectures (VSA) framework to construct semantic scene representation. Our method uses semantic segmentation of an image to extract objects and their relations and applies VSA operations to form semantic scene representation. For this, an optional usage of the depth map was considered, which showed promising results. The effectiveness of our approach is demonstrated through extensive experiments on the open large-scale datasets: the indoor HPointLoc dataset built in the Habitat simulation environment and the outdoor Oxford RobotCar dataset. The proposed solution significantly improves the quality of the place recognition.

I. Introduction

The solution to the place recognition problem is an essential part of approaches to the global localization of intelligent agents, particularly, robots. The use of onboard camera images for this purpose makes such solutions simpler and cheaper. Furthermore, the advent of affordable RGB-D cameras and fast and high-quality algorithms for semantic segmentation makes it possible to use information about the objects' presence and spatial relationships with various semantic categories.