Synthesizing with Diffusion model for improving medical image segmentation performance

Diffusion probabilistic models (DPMs) have attracted much attention in the field of computer vision by demonstrating superior image generation capabilities. However, in the field of medical images, the scarcity of annotations often poses a challenge for the application of deep learning-like methods. Motivated by the success of DPMs, we present Medical Image Segmentation Augmentation Diffusion Model (MEDSAD), a DPM and mask-based medical image generation model designed for general medical image generation tasks to alleviate the problem of scarcity of images with annotation. MEDSAD leverages a simple annotation to generate paired medical images, thereby enhancing the model’s data exploration capabilities and improving segmentation performance in downstream segmentation tasks. To exert greater control over the texture generated from medical images in MEDSAD, we introduce the Texture Style Injection (TSI) mechanism, which restricts the model from generating textures rather than randomly. Additionally, we propose a Feature Frequency Domain Attention (FFDA) module to mitigate the adverse effects of high-frequency noise components during this process. We validate the performance enhancements achieved by MEDSAD on two medical segmentation tasks involving magnetic resonance (MR) and ultrasound (US) image modalities for breast tumor and brain tumor segmentation. The results demonstrate that MEDSAD outputted high-quantity synthetic pair medical images from given annotation, delivering the most substantial performance improvement even with a limited number of samples in the downstream segmentation task. These findings underscore the generalization and efficacy of the proposed model.