Segmentation of Remote Sensing Data with Missing Modalities Through Prototype Knowledge Distillation

Segmentation of remote sensing data is a critical task in various environmental and geospatial applications. However, the presence of missing modalities, such as the Digital Elevation Model (DEM), poses significant challenges to achieving high segmentation accuracy. In this paper, we propose a novel approach for addressing this issue using Prototype Knowledge Distillation. Our methodology involves training a teacher model with access to all available modalities, including DEM, to generate high-quality segmentation results. Subsequently, we train a student model that performs segmentation without access to the DEM modality. The teacher model distills its learned knowledge into the student model through prototype representations, ensuring that the student model can achieve comparable or better performance despite the missing modality. Experimental results demonstrate the effectiveness of our approach, showing significant improvements in segmentation accuracy over baseline methods that do not utilize knowledge distillation. This work paves the way for robust segmentation of remote sensing data in scenarios where certain modalities are unavailable, enhancing the applicability and reliability of remote sensing analyses.