Semantic-Guided Voting with Segmentation for Visual Localization

Scene coordinate regression has significantly enhanced visual localization by segmenting landmark patches and aggregating votes for landmark points within these patches. However, this approach neglects dynamic semantic objects and patch-level semantic inconsistencies, leading to inaccuracies in the voting process. In this paper, we present a novel visual localization framework guided by semantics. Our framework introduces two critical modules: the Pruning Semantically Dynamic Object module, which leverages semantic segmentation to reduce interference from dynamic semantic objects, and the Eliminating Semantic Label Inconsistencies module, aimed at mitigating patch-level semantic inconsistencies. Comprehensive tests on 7-Scenes and Cambridge datasets exhibit the superior performance of our approach. Across both datasets, our method outperforms the scene coordinate regression technique, validating the effectiveness of our proposed framework.