DiffVector: Boosting Diffusion Framework for Building Vector Extraction From Remote Sensing Images

Building vector maps play an essential role in many remote sensing (RS) applications, thereby boosting the deep learning (DL)-based automatic building vector extraction methods. These approaches have achieved pleasant overall accuracy, but their predict-style framework struggles with perceiving subtle details within a tiny area, such as corners and adjacent walls. In this study, we introduce a denoising diffusion framework called DiffVector to generate representations for direct building vector extraction from the RS images. First, we develop a hierarchical diffusion transformer (HiDiT) to conditionally generate robust representations for detecting nodes and extracting corresponding features. The conditions of HiDiT are multilevel boundary attentive maps encoded from input RS images through a topology-concentrated Swin Transformer (TCSwin). Subsequently, an edge-biased graph diffusion transformer (EGDiT) takes extracted node features as conditions to produce new visual descriptors for the adjacency matrix prediction. In EGDiT, we replace the standard self-attention (SA) operation with an edge-biased attention (EBA) to inject edge information for training stabilization. Furthermore, given typical challenges of training difficulty and weak perceptive ability in convectional diffusion paradigms, we conduct an isomorphic training strategy (ITS), ensuring that the training procedures of both HiDiT and EGDiT precisely mirror the inference phase. Quantitative and qualitative experiments have evidently demonstrated that DiffVector can achieve competitive performance compared with existing modern approaches, especially in the metrics assessing topology quality.

I. Introduction

Building vector maps are structured as directional graphs recording footprints, connectivity, and associated attributes, which supports distinct advantages of lossless scalability, convenient topological analysis, free attribute edition, and low storage cost. These characteristics establish the pivotal role of building polygon data in many remote sensing (RS) and geographic information system (GIS) applications, such as population density estimation, disaster management, and urban planning [1], [2], [3]. The brisk demand booms studies on automatic polygonal buildings extraction from RS images in order to replace time-consuming and labor-intensive manual and semiautomatic production. Especially in recent years, deep learning (DL) methods offer promising solutions to automatic polygonal building map extraction, and they can be grouped into three paradigms, namely, segmentation-based, counter-based, and node-based.