Knowledge-based reliability assessment of models with application to risk stratification

This paper proposes a framework for developing reliability metrics for model predictions based on a priori clinical knowledge. The approach relies on rule-based knowledge along with selected features under the form of risk factors. Moreover, personalization of predictions plays a crucial role for grouping individuals into specific groups or profiles based on their intrinsic characteristics. Appropriate rules according to the individuals’ profile are derived and used within the framework. The proposed approach is validated considering the Global Registry of Acute Coronary Events risk model and applied to a dataset from the Centro Hospitalar da Universidade de Coimbra, with focus on risk assessment. Results obtained using the proposed approach indicate that by using those metrics to assess reliability it is possible to identify particular cases in which GRACE presents almost predictable outcomes.