The Accuracy Performance of Semantic Segmentation Network with Different Backbones

With the fast improvement of classification networks, many of these networks are being in use as backbones of semantic segmentation networks to improve the accuracy. Using different classification networks as the backbone of the same semantic segmentation network may show different accuracy performance. This paper selected the sandstone dataset and self-driving cars dataset to compare the accuracy performance differences of VGG-16, ResNet-34, and Inceptionv3 as the backbone of UNet, where the original encoder of the UNet is replaced by a backbone. The three backbone networks are imported from Segmentation Models library, and they have weights trained on ImageNet dataset. The best accuracy performance of the semantic segmentation network on the sandstone dataset is when VGG-16 is used as the backbone, it achieved 76.22% MIoU. On the other hand, the highest accuracy performance of the semantic segmentation network on self-driving cars dataset is 75.47% MIoU, achieved when Inceptionv3 is used as the backbone. However, the accuracy is improved when using all the three backbones with both datasets, compared to the accuracy performance of the UNet without using any backbone network.