Discovering Hidden Knowledge and Optimizing the Model by Analyzing Linear Regression Assumptions

Linear regression is the most frequently used regression analysis due to its simplicity in predicting and forecasting. However, because of its parametric approach, it is necessary to test five assumptions before using this technique on a given dataset. Two of these assumptions are critical, such as linearity and multicollinearity. Linearity has significance for representing patterns; whereas, multicollinearity reduces the model’s interpretability. If the negligible level of linearity and the most extreme level of multicollinearity are known before applying regression analysis, it saves significant computing time. This study centers around the level of the greatest linearity and negligible multicollinearity expected for model success. Ordinary least squares (OLS) regression analysis has been done for this purpose. A thorough investigation of ten UCI datasets reveals that at least 70% linearity is required for the model to perform well, while a maximum of 45% multicollinearity is acceptable. Furthermore, feature engineering is used to enhance the efficiency of the model by prioritizing linearly associated significant features with the target variable and contains no or little multicollinearity concerns. This implies the final model is very close to the assumptions, lead to upgrade the performance of the model.