Modality-shared MRI Image Translation Based on Conditional GAN

Multimodal MRI images are often necessary for precise clinical diagnosis and the development of high-performance intelligent medical image systems. However, it is expensive and often difficult to obtain sufficient registered multimodal MRI images. One way to alleviate this issue is MRI image translation which aims to generate images of another modality based on images of one modality. This paper proposes a novel image translation framework that can translate images of any MRI modality to any other one. The core part of the framework is a conditional generative adversarial network (CGAN), where unlike commonly used conditions such as one-hot vectors, an image-specific condition can be generated based on the gradient-weighted class activation mapping (Grad-CAM), which can highlight the parts of the input image that need more attention during the translation process. In addition, to translate the lesion information in the MRI images, a modality-shared segmentation model is proposed to extract the lesion information, which is then novelly embedded into the image translation process. Comprehensive experiments on different multi-modality MRI datasets demonstrated the effectiveness of the proposed translation approach, achieving better performance compared to state-of-the-art methods.