DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients

Line segments are ubiquitous in our human-made world and are increasingly used in vision tasks. They are complementary to feature points thanks to their spatial extent and the structural information they provide. Traditional line detectors based on the image gradient are extremely fast and accurate, but lack robustness in noisy images and challenging conditions. Their learned counterparts are more repeatable and can handle challenging images, but at the cost of a lower accuracy and a bias towards wireframe lines. We propose to combine traditional and learned approaches to get the best of both worlds: an accurate and robust line detector that can be trained in the wild without ground truth lines. Our new line segment detector, DeepLSD, processes images with a deep network to generate a line attraction field, before converting it to a surrogate image gradient magnitude and angle, which is then fed to any existing handcrafted line detector. Additionally, we propose a new optimization tool to refine line segments based on the attraction field and vanishing points. This refinement improves the accuracy of current deep detectors by a large margin. We demonstrate the performance of our method on low-level line detection metrics, as well as on several downstream tasks using multiple challenging datasets. The source code and models are available at https://github.com/cvg/DeepLSD.

1. Introduction

Line segments are ubiquitous in human-made environments and encode the underlying scene structure in a compact way. As such, line features have been used in multiple vision tasks: 3D reconstruction and Structure-from- Motion (SfM) [17], [32], [34], Simultaneous Localization and Mapping [13], [15], [25], [37], [62], visual localization [14], tracking [38], vanishing point estimation [49], etc. Thanks to their spatial extent and presence even in textureless areas, they offer a good complement to feature points