MemXCT: Design, Optimization, Scaling, and Reproducibility of X-Ray Tomography Imaging

This work extends our previous research entitled “MemXCT: Memory-centric X-ray CT Reconstruction with Massive Parallelization” that was originally published at SC19 conference (Hidayetoğlu et al., 2019) with reproducibility of the computational imaging performance. X-ray computed tomography (XCT) is regularly used at synchrotron light sources to study the internal morphology of materials at high resolution. However, experimental constraints, such as radiation sensitivity, can result in noisy or undersampled measurements. Further, depending on the resolution, sample size and data acquisition rates, the resulting noisy dataset can be in the order of terabytes. Advanced iterative reconstruction techniques can produce high-quality images from noisy measurements, but their computational requirements have made their use an exception rather than the rule. We propose a novel memory-centric approach that avoids redundant computations at the expense of additional memory complexity. We develop a memory-centric iterative reconstruction system, MemXCT, that uses an optimized SpMV implementation with two-level pseudo-Hilbert ordering and multi-stage input buffering. We evaluate MemXCT on various supercomputer architectures involving KNL and GPU. MemXCT can reconstruct a large (11K×11K) mouse brain tomogram in 10 seconds using 4096 KNL nodes (256K cores). The results presented in our original article at the SC19 were based on large-scale supercomputing resources. The MemXCT application was selected for the Student Cluster Competition (SCC) Reproducibility Challenge and evaluated on a variety of cloud computing resources by universities around the world in the SC20 conference. We summarize the results of the top-ranked SCC Reproducibility Challenge teams and identify the most pertinent measures for ensuring the reproducibility of our experiments in this article.

1 Introduction

X-ray computed tomography (XCT) is a widely used nondestructive 3D imaging technique for observing and understanding the internal morphology of samples. Synchrotron light sources, such as the Advanced Photon Source (APS), can provide high-brilliance X-rays that enable tomographic imaging of centimeter sized samples at sub-micrometer ($\mu$m) spatial resolution. Such experiments can generate from a few GBs to TBs of data volumes in a short time period with the typical pixelated detectors that can run at 16 GB/s [2]. However, the quality of data collected from CT experiments depends heavily on factors such as radiation exposure time (dose) and target spatial resolution. Much effort has been devoted to develop and implement advanced reconstruction algorithms to improve image quality when collected data are noisy or imperfect (e.g., due to limited dose).