PUW-Feat: A Progressive and Unified Method for Weakly Supervised Local Feature Learning

Local feature extraction is a fundamental module in computer vision. Weakly supervised learning methods have more convenience of collecting datasets, but they have still not get proper trade-off among training costs, accuracy and speed. In this paper, we propose PUW-Feat, a Progressive and Unified Weakly supervised learnable local Feature extractor. We design a progressive describe-then-detect learning pipeline to save training costs, which partly decouples the training process yet ensures its consistency by sharing the loss function structure. We build a unified keypoint location training framework which can predict keypoint locations by a learnable network branch to avoid slow post-process, thus we increase speed while keep accuracy. Our method achieves the best balance on training costs, accuracy and real-time performance in experiments on different tasks.