Lightweight Cross-Modal Information Measure and Propagation for Road Extraction From Remote Sensing Image and Trajectory/LiDAR

Recent studies have confirmed that GPS trajectory can effectively assist in achieving more accurate road extraction from remote sensing images (RSIs). Therefore, lots of efforts focus on designing effective multimodal fusion strategies for GPS trajectory and RSI modalities. However, there are still some limitations, e.g., the fusion structures are complex and hinder further improvements. Moreover, the negative impact of redundant information in various modalities is commonly ignored. This article aims to design a simple yet effective fusion strategy for GPS trajectory and RSI modalities to address the above issues. Inspired by the network pruning algorithm, we design a cross-modal information propagation (CMIP) mechanism. CMIP utilizes the scaling factors and sparse constraint to distinguish the redundant information of a certain modality that is directly replaced with corresponding information of another modality. We improve the widely used
$\mathcal {L}$
1 sparse constraint and propose a novel information balanced constraint which is added on the scaling factors to better identify and prune redundant channels. Embedded in the CMIP mechanism, a multimodal information propagation network, CMIP network (CMIPNet) is proposed, which can fully explore the complementarities between different modalities to accurately locate roads, especially roads with noise or incomplete information of a certain modality. Since the CMIP is parameter-free and self-adaptive, CMIPNet is lightweight and easy to deploy. The parameter number of CMIPNet can be comparable to single-modal models, which is about one-third of the existing multimodal models. Extensive experiments are performed to demonstrate that CMIPNet outperforms the previous single- and multimodal road extraction methods.