Accelerating Deeply Cascaded Convolutional Networks for MRI Reconstruction via Pixel-Unshuffle Caused Feature Squeezing

Network acceleration is very important for Magnetic Resonance Imaging (MRI) reconstruction, as the device-specific retraining of a given network is usually necessary to overcome the scanner transfer problem. We propose a novel framework, namely PU-Network-PS (UNS), to accelerate the Deeply Cascaded Convolutional Networks (DCCNs) for MRI reconstruction, where Pixel Unshuffle (PU) and Pixel Shuffle (PS) are imposed at the two ends of DCCNs, respectively. Our UNS accelerates DCCNs by first decomposing the zero-filled MRI images of DCCNs into several sub-images via using PU, which are dubbed PU-channels and will make DCCNs mainly run in a low-resolution space and thus obtain high computation efficiency. However, directly using PU-channels might lead to unstable performance due to the lost spatial correlations between PU-channels. We discovered that the PU-channels can be organized into different groups according to their mutual similarities and presented a novel Multi-Scale Cross Group Convolution to fully fuse their complementary information. We further integrated into UNS the ensemble learning scheme in the PS stage to boost performance. Experiments on the IXI dataset demonstrated that our UNS can enhance both the reconstruction performance and the running efficiency for both simple and complex DCNN models. Specially, the training time for DuDoRNet can be greatly reduced from three days to about 20 hours with small performance enhancements by using two NVIDIA RTX 2080Ti GPUs.