Adaptive Deep Metric Ensemble Learning with Consensus

To measure semantic similarity of data pairs using multiple metrics has received much attention. It is natural to construct an ensemble consisting of base learners for modeling distinct properties of data distribution in the embedding space. However, previous works fail to correlate data subset generation with the performance of base learners and ignore possibly large deviations of different metrics on measuring the same data pairs. To address these issues, we propose the Consensus-aware Adaptive Ensemble (CAE) framework for deep metric learning. CAE adaptively provides data subsets for base learners which are empowered with both diversity and consensus. For each learner, the samples of the classes having larger intra-class distances in the embedding space will form the subset periodically. Moreover, to diversify learners, we employ determinant point process loss to make them capture various semantics of data distributions, promoting the generalization ability of ensemble. Meanwhile, different learners are expected to agree on measuring the same data pairs, so we develop the consensus loss to guide learners to reach a consensus as much as possible. Extensive experiments on several benchmarks demonstrate that CAE achieves state-of-the-art performance, validating its effectiveness.