No-gold-standard Evaluation of Quantitative Imaging Methods: Studying the Impact of Model Mismatch

Objective evaluation of quantitative-imaging (QI) methods with patient data is highly desirable but often hindered by the lack of gold standards. To address this challenge, no-gold-standard evaluation (NGSE) techniques have been proposed. The efficacy of these techniques has been demonstrated in multiple studies. However, these techniques, by virtue of being model based, makes certain assumptions which, while reasonable and justified, could potentially be violated in clinical settings. Evaluating the sensitivity of these NGSE techniques to these assumptions is important for clinical translation of this technique. To achieve this goal, we conducted multiple studies where we investigated the sensitivity of a recently proposed NGSE technique to three sources of model-mismatch: (a) non-linearity between true and measured values (b) mismatch between true and assumed distributions of true values and (c) correlation between true values. Our results showed that the NGSE technique was relatively insensitive to mismatches in distributions between the true and measured values and to correlation between true values. Further, the NGSE technique was observed to yield reliable performance for certain deviations from linearity but could be unreliable for larger degrees of nonlinearity between measured and true values. Overall, this study demonstrates the NGSE technique is robust to multiple types of model mismatch. The results provide confidence for clinical applicability of the NGSE technique.