Learning to Reduce Scale Differences for Large-Scale Invariant Image Matching

Most image matching methods perform poorly when encountering large scale changes in images. To solve this problem, we propose a Scale-Difference-Aware Image Matching method (SDAIM) that reduces image scale differences before local feature extraction, via resizing both images of an image pair according to an estimated scale ratio. In order to accurately estimate the scale ratio for the proposed SDAIM, we propose a Covisibility-Attention-Reinforced Matching module (CVARM) and then design a novel neural network, termed as Scale-Net, based on CVARM. The proposed CVARM can lay more stress on covisible areas within the image pair and suppress the distraction from those areas visible in only one image. Quantitative and qualitative experiments confirm that the proposed Scale-Net has higher scale ratio estimation accuracy and much better generalization ability compared with all the existing scale ratio estimation methods. Further experiments on image matching and relative pose estimation tasks demonstrate that our SDAIM and Scale-Net are able to greatly boost the performance of representative local features and state-of-the-art local feature matching methods.

I. Introduction

Establishing pixel-level correspondences between two images is an essential basis for a wide range of computer vision tasks such as visual localization [1], [2], 3D scene reconstruction [3] and simultaneous localization and mapping (SLAM) [4]. Such correspondences are usually estimated by sparse local feature extraction and matching [5], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16]. A local feature consists of a keypoint and a descriptor. But the scale invariance of both existing keypoint detectors and descriptors is not enough to deal with large scale changes [17]. Few inlier correspondences can be established by matching local features under the circumstances of large scale changes in images, which is called as the scale problem of local features in this paper. If the scale difference between two images is small, we call that the two images are at related scale levels in scale space [18], [19].