SR2Net: A Separatable Reconstruction Residual Network Based on Channel Attention for PET and MRI Image Fusion

Image fusion is an important branch of medical image processing and analysis. The main steps of image fusion are: acquiring and preprocessing data from source images, extracting and selecting relevant features, choosing suitable fusion methods and algorithms, generating fused results, and evaluating them. While significant progress has been made in feature extraction and reconstruction, the fusion strategy remains an ongoing exploration. Therefore, we propose an innovative fusion strategy, namely the Three-Branch SR
²
Fusion Strategy. We adopt a two-stage training approach, initially training an encoder and a nested-connected decoder as an auto-encoder to extract multi-modal deep features. Subsequently, our trained SR
²
fusion network integrates the extracted deep features. The initialized features are fed into a Separation Reconstruction Unit (SRU) network, branching into two pathways: a channel attention network and a convolutional neural network. This is designed to reduce the redundancy of spatial features while preserving the structural information of the source images. Also, the initialized features from MRI and PET images are concatenated and fed into the third branch. In addition, to prevent feature loss and retain more detailed information, we introduce three recurrent residual modules. Finally, the features obtained from three branches are aggregated and the decoder is employed to reconstruct the ultimate fused images. The experimental results conducted on the public dataset demonstrate the superiority of our approach in both subjective and objective evaluations compared to other advanced methods.