An Effective Dual-Scale Hybrid Encoder Network for Medical Image Segmentation

Medical image segmentation’s accuracy is crucial for clinical analysis and diagnosis. Despite progress with U-Net-inspired models, they often underuse multi-scale encoding layers crucial for enhancing detailing visual features and overlooking the importance of merging multi-scale features within the channel dimension to enhance decoder complexity. To address these limitations, we introduce a dual-scale hybrid encoder network DSENet for medical image segmentation. Our network design is characterized by the strategic employment of dual-scale convolutional kernels at each encoder level, integrating the robust feature extraction capabilities of CNNs with the contextual awareness of Transformer models. This synergy enables the precise capture of both granular and broader context features throughout the encoding phase. We further enhance the model by integrating a channel attention fusion (CAF) mechanism within the skip connections. This innovation effectively integrates dual-scale features, subsequently integrating them with the same-level decoder feature map, thereby reinforcing the feature representation. To refine the predicted segmentation, we employ a novel strategy that merges dual-scale feature maps from the initial encoder stage with the segmentation map through a cascade operation. The output fusion feature map is then processed by a self-attention Transformer structure, ensuring a meticulous refinement of the segmentation output, preserving essential details, and enhancing segmentation accuracy. Our proposed DSENet has been evaluated on three distinct medical image datasets, and the experimental results demonstrate that it achieves more accurate segmentation performance and adaptability to varying target segmentation, making it more competitive compared to existing SOTA methods.