Learning Rich Multimodal Representation for Robust Land Cover Classification in Fog

When drones perform remote sensing tasks, such as land cover classification (LCC) in foggy conditions, sensor failures are frequently encountered, making it essential to utilize multimodal data. Heterogeneous feature spaces can be leveraged to obtain a comprehensive representation of specific locales, increasing the discriminability of terrestrial covers. However, existing multimodal fusion methodologies fail to fully capitalize on the unique attributes of individual modalities. Additionally, many approaches fall short in robust feature extraction and fusion from multimodal inputs, leaving room for improvement in classification accuracy. We introduce a cascading backbone network designed to adaptively fuse and mine the spatial contributions of multimodal data inputs. Signal-to-noise ratio (SNR) is used to guide attention toward salient spatial features within individual modalities. The proposed context representation enhancement module (CREM) enables the classification of spatial regions with more discriminative attributes, while mitigating the impact of irrelevant noise in digital surface models (DSMs). Additionally, we engineer a cross-modal fusion module (CMFM) to recalibrate and refine multimodal feature tensors, exploring cross-modal complementarity. Experimental validation on three distinct datasets corroborates the robustness for our framework applied in LCC, substantiating its efficacy in semantic segmentation tasks in fog-corrupted environments.