Evaluating SHAP’s Robustness in Precision Medicine: Effect of Filtering and Normalization

Local interpretation of explainable AI, SHAP (SHapley Additive exPlanations), in disease classification problems offers significant feature scores for each sample, potentially identifying precision medicine targets. Tailoring treatments based on individual genetic and molecular targets can enhance therapeutic outcomes while minimizing side effects. However, the suitability of SHAP's local interpretation at the patient level remains uncertain. It generates different sets of patient-specific genes in various runs, even with consistent overall accuracies. This uncertainty challenges the reliability of SHAP’s local interpretations for precision medicine applications. Not only that, different filtering criteria and normalization techniques may influence the contribution scores of patient-specific features. To validate our hypothesis, SHAP was applied to machine learning algorithms from different genres to identify patient-specific feature contributions from the breast cancer subtype classification problem. The program underwent multiple runs to assess the robustness of SHAP.Our study demonstrates that shallow machine learning algorithms, like Logistic Regression, consistently provided stable and reliable results across multiple runs. In contrast, complex machine learning models like XGBoost and MLP exhibited inconsistencies across different runs. Moreover, we found that data normalization techniques, particularly z-score and min-max normalization, had a minimal effect on the performance of XGBoost models. Our study also shows that the accuracy scores of complex machine learning models remained relatively constant across different runs but produced different sets of patient-specific features. In conclusion, our findings underscore the importance of selecting appropriate filtering and normalization techniques, given the variability in SHAP results across different runs. Our study indicates that combining SHAP with shallow machine learning algorithms yields more stable and dependable results compared to complex machine learning approaches.