OIF-Net: An Optical Flow Registration-Based PET/MR Cross-Modal Interactive Fusion Network for Low-Count Brain PET Image Denoising

The short frames of low-count positron emission tomography (PET) images generally cause high levels of statistical noise. Thus, improving the quality of low-count images by using image postprocessing algorithms to achieve better clinical diagnoses has attracted widespread attention in the medical imaging community. Most existing deep learning-based low-count PET image enhancement methods have achieved satisfying results, however, few of them focus on denoising low-count PET images with the magnetic resonance (MR) image modality as guidance. The prior context features contained in MR images can provide abundant and complementary information for single low-count PET image denoising, especially in ultralow-count (2.5%) cases. To this end, we propose a novel two-stream dual PET/MR cross-modal interactive fusion network with an optical flow pre-alignment module, namely, OIF-Net. Specifically, the learnable optical flow registration module enables the spatial manipulation of MR imaging inputs within the network without any extra training supervision. Registered MR images fundamentally solve the problem of feature misalignment in the multimodal fusion stage, which greatly benefits the subsequent denoising process. In addition, we design a spatial-channel feature enhancement module (SC-FEM) that considers the interactive impacts of multiple modalities and provides additional information flexibility in both the spatial and channel dimensions. Furthermore, instead of simply concatenating two extracted features from these two modalities as an intermediate fusion method, the proposed cross-modal feature fusion module (CM-FFM) adopts cross-attention at multiple feature levels and greatly improves the two modalities’ feature fusion procedure. Extensive experimental assessments conducted on real clinical datasets, as well as an independent clinical testing dataset, demonstrate that the proposed OIF-Net outperforms the state-of-the-art methods.

I. Introduction

Positron emission tomography (PET) is an in vivo functional imaging technique that uses the intravenous injection of target-specific tracers to reveal the metabolism of a live organism. A wide range of nuclear medicine applications are based on PET imaging, including early disease detection [1] and postoperative staging diagnosis [2] in patients, tumor diagnosis [4] and neurological diagnosis [6]. The accumulated imaging count of radiotracers caused by PET acquisition steps potentially poses a secondary cancer risk for the pediatric population [7]. Furthermore, a long scanning duration may have a detrimental impact on patients’ comfort. Reducing the radiotracer [8] dose and decreasing the PET scanning time [9] in clinical applications will greatly benefit patient health. Despite the improvements achieved regarding the hardware characteristics of PET imaging, low-count PET still hinders the diagnoses of doctors due to poor image quality issues, including high statistical noise, low contrast levels, high signal-to-noise ratios (SNRs) and orientation biases [10]. Thus, the use of a proper PET image postprocessing method to enhance low-count PET imaging quality is necessary.