A Deep Texture Metric for PET Image Quantification and Image Synthesis

Fluorodeoxyglucose (FDG) Positron Emission Tomography (PET) serves as a cornerstone in functional imaging. PET imaging reveals metabolic abnormalities before morphological alterations occur. The standardized uptake value is a common metric to semi-quantify PET images. Visual scoring of PET image heterogeneity has demonstrated correlations with tumour characteristics. Quantification of this spatial variation in FDG distribution via its shape and texture has proven to be a fertile area of research under the umbrella term "Radiomics." Deep learning with convolutional neural networks (CNN) serves as large feature extractors with proven ability in tumour type classification in PET imaging. Deep learning in PET imaging has also been utilized for image enhancement, reconstruction, segmentation and image synthesis. In this work, we quantify the feature distribution of neural activations of each layer in a CNN following propagation through the network of a PET image. The Gramian matrix of the activation functions quantifies the neural representation of the CNN and thus serves as a measure of visual perception. Image synthesis using gradient descent on the target PET image Gramian matrix allows visualization of the texture. This metric is tested in PET imaging data of a recently developed texture phantom and a single primary tumour from a non-small-cell lung cancer (NSCLC) patient. It is demonstrated that the summation of the Gramian decreases with decreasing homogeneity. The visual appearance of the synthesized PET images closely resembles that ground truth PET images, both in the texture phantom and clinical image.