Learning General Feature Descriptor for Visual Measurement With Hierarchical View Consistency

Learning-based feature descriptors have been dominantly popular for their notable performance on feature matching tasks along with the rapid development of convolutional neural networks (CNNs). However, existing popular learning-based methods predict discriminative description solely using the high-level features from the last layer of deep CNNs while neglecting the rich complementary clues hidden in intermediary multilevel features, which could further promote the discriminative power by introducing the implicit hierarchical comparison into descriptor space. This hinders the optimization of learned descriptors and limits their performance on real-world visual measurement tasks. In this regard, we propose hierarchical view consistency (HVC) for fully leveraging the complementary information of multilevel features. Specifically, we first present a novel multiviewer neural network (MVNet), which benefits from multiple viewers with local-to-global receptive fields and efficiently generates dense descriptions in a coarse-to-fine manner. Next, we introduce the HVC, i.e., ensuring consistent yet diverse hierarchical features between views, to encourage viewers to encode as hierarchical features as possible while increasing the hierarchical similarity for reliable matches. With our proposed triplet training strategy, MVNet leverages the rich hierarchical complementary clues in multilevel features and efficiently predicts strong discriminative descriptions. Our experiments on feature matching and challenging visual measurement tasks of visual localization and visual 3-D reconstruction demonstrate that our proposed descriptor is efficient and generalizes well to various scenarios.

I. Introduction

Feature description is the corner stone of numerous visual measurement tasks: geometric visual reconstruction [well known as structure from motion (SfM)] [1], simultaneous localization and mapping (SLAM) [2], [3], visual localization [4], detection and tracking [5], and so on. Early efforts paid attention to the design of compact, general, and efficient handcrafted descriptors relying on local intensity comparison [6]–[9] and gradient statistics [10]–[13]. Although some well-designed handcrafted descriptors, such as SIFT [10] and SURF [12], have been widely used in various visual tasks for their simplicity and practical performance, they disregard useful patterns hidden in the data and, thus, fail to provide reliable matches in a complex environment scenarios [14].