E-DM: Evaluating Diffusion Model by Conformal Prediction

Diffusion models, as new score-based generative models, have drawn increasing attention by achieving state-of-the-art performance on different domains and tasks. The majority use signal to noise methods (e.g., PSNR, SSIM) as evaluation metrics because diffusion models generally learn through a reversible denoising process. However, evaluating the diffusion models with comprehensive consideration is still a pressing challenge, especially for the outcome reliability that involves sensitive information. Specifically, users in the healthcare domain also ask for trustworthiness and image quality. To address this issue, we introduce the Conformal Prediction Set into diffusion models as an uncertainty quantification method, namely E-DM, providing uncertainty-controlled results without compromising image quality. Conformal Prediction allows us to estimate a confidence set with guaranteed coverage for any finite population, thereby providing a reliable monitor for any black-box predictor. In this research, we illustrate our approach by improving the trustworthiness degree of a Denoising Diffusion Probabilistic Model (DDPM) for super-resolution tasks on Magnetic Resonance Imaging (MRI) images.