Whole-Body Bone Scan Segmentation Using SegFormer

Semantic segmentation can help in everyday life, especially in the medical field, to help detect cancer metastasis at an early stage. In semantic segmentation, CNN-based approaches have been known to dominate the semantic segmentation field, such as FCN and DeepLabv3+. The success of the Transformer approach in the Nature Language Processing (NLP) area triggered many researchers to use the Transformer approach in solving semantic segmentation problems, so the Vision Transformer (ViT) was born. ViT accepts images as patches to produce local and global attention, unlike the convolution approach. One ViT-inspired model, SegFormer, combines a Hierarchical Transformer encoder component to generate low-resolution fine features that focus on capturing small detailed, fine-grained information such as edges, corners, and local patterns, and high-resolution coarse features that focus on capturing more general and global characteristics of a scene, and a lightweight All-MLP decoder to combine those multi-level features to produce the final semantic segmentation mask. This allows SegFormer to capture local and global contextual information from an image. This paper proposed the SegFormer model to perform semantic segmentation of bone scan images into 12 classes based on bone regions. As a result, by comparing FCN and DeepLabv3+ convolution approaches, SegFormer outperforms both models with the highest mIoU value of 77.86%.