A Comprehensive Study of Regression Analysis and the Existing Techniques

In many different sciences, including medicine, engineering, and observational studies, the investigation of the relationship between variables, i.e., dependents, and independents, is defined as research objectives. Employing statistical methods to achieve the relationship between variables is very time-consuming or costly in many scenarios and does not provide practical application. Therefore, numerous machine learning algorithms have been introduced with the advancement of science and inspiration from nature to perform regression and modeling. Machine learning models have been able to have an excellent position in this field and provide admirable results. This paper examines and compares various regression models and machine learning algorithms. The selected techniques include multiple linear regression (MLR), ridge regression (RR), least absolute shrinkage and selection operator (LASSO) regression, multilayer perceptron (MLP), radial basis function (RBF), decision tree (DT), support vector regression (SVR), and k-nearest neighbors (KNN). In addition, many evaluation metrics are employed to further investigate and compare the performances of the selected algorithms on a couple of datasets. Our simulation results further demonstrate the significant superiority and accuracy of the presented work.