Comparing the robustness of U-Net, LinkNet, and FPN towards label noise for refugee dwelling extraction from satellite imagery

Extracting footprints of refugee dwellings from satellite imagery supports dedicated humanitarian operations. Recently, deep-learning-based approaches have been proved to be effective for this task. However, such research is still limited due to the lack of cleaned labels for supervised-learning-based models. This research compares the performance of noisy labels from past humanitarian operations and cleaned labels by manual annotation in three classical deep learning architectures (U-Net, LinkNet and Feature Pyramid Network (FPN)) and twelve backbones (VGG16, VGG19, ResNet-18, ResNet-34, DenseNet-121, DenseNet-169, Inception-v3, InceptionResnet-v2, MobileNet-v1, MobileNet-v2, EfficientNet-B0, EfficientNet-B1). The results turn out that even though cleaned labels outperform noisy labels, the noisy labels have a high potential to replace cleaned labels because producing cleaned labels requires much more time, and predicted footprints of models trained with noisy labels are promising in humanitarian applications. Besides, the performance of the selected architectures and backbones is similar in general. Overall, FPN with VGG16, LinkNet with DenseNet-121, and U-Net with EfficientNet-B0 outperform other combinations. If considering both accuracy and training time, U-Net with VGG16 and LinkNet with ResNet-18 could be two alternatives for future research.