Impact of Outliers and Dimensionality Reduction on the Performance of Predictive Models for Medical Disease Diagnosis

Numerical electronic health records often come with numerous features and outliers. These features are usually indicators of medical diseases. To prevent poor model performance associated with the curse of dimensionality, these features must be reduced with only the most important ones retained. Also, outliers may result in poor generalization and performance of a machine learning model. This study investigates the impact of dimensionality reduction and outliers on machine learning models for medical disease diagnosis. After fitting datasets to the models, the impact of outliers and dimensionality reduction was evaluated using performance evaluation metrics. The accuracy for all models across the Wisconsin breast and heart dataset illustrates a minimum of 1.5% increase compared to when PCA was not implemented. Also, the precision, recall, and f1 score values show significant improvement. The naïve Bayes model recorded an accuracy of 73.6% in the absence of outliers and 72.7% in the presence of outliers when the Pima Indian diabetes dataset was used. This is less than 1% improvement in the model’s performance. The result indicates that dimensionality reduction significantly impacts the performance of models in diagnosing breast cancer and heart disease. Furthermore, outliers did not significantly impact the model’s performance when using Pima Indian and stroke datasets. Finally, this study depicts that problems associated with the curse of dimensionality persist with machine learning models in diagnosing medical diseases.