Dimensionally Unified Metric Model for Multisource and Multiview Scene Matching

The core challenge of multiview scene matching is to effectively extract features from multiview images, which is a key factor in achieving robust scene matching. This article delves into methodologies for enhancing the multiview robustness of scene matching, with the following key contributions: 1) this article proposes the metric feature consistency principle which emphasizes the necessity of implementing accurate correspondence between drone and satellite images. To verify the principle, consistent feature enhancement for channel, spatial, and hybrid dimensions is explored. As a counterexample, the shuffle operation is used to break the consistency of dimension semantic information. Experiments show that the mAP is reduced by about 20% after breaking the consistency relationship of metric features; 2) built upon the principle, this article introduces a scene-matching framework named dimensionally unified metric model (DUMM). Utilizing the multidimensional feature enhancement model effectively enhances the correspondence of the dual-branch features within the siamese network; and 3) this article is the first to introduce the feature dimension and frequency-domain feature. By the upper and lower bounds of the cosine transform function, the negative effects of cross-view variations can be effectively mitigated, thus enhancing the overall robustness. The introduction of frequency features improves the mAP by 2.54% compared with the method of improving the dimensional consistency of
$H \times W \times C$
, verifying the positive impact of the fusion of frequency domain features on enhancing the robustness of scene matching. Nevertheless, it remains imperative to adhere to the principle of feature dimension consistency across all additional frequency dimensions.