Data Transformation in Machine Learning: Empirical Analysis

Data transformation is a critical yet often overlooked preprocessing step in many data science pipelines. A right transformation can significantly improve the performance of the learning algorithm. Conversely, a wrong transformation can skew the data leading to adverse results. To better understand the effects of data transformation and provide best-practice recommendations, this research aims to analyze the efficacy of various data transformation techniques. We consider several commonly used transformation techniques and compare their effects on the learning model. In particular, normalization, unitization, logarithmic, square root, and arctangent transformations are compared. The transformations are applied on continuous-valued data before fitting linear regression. The accuracy of the resulting model is used as the measure of transformation efficacy. The results show that logarithmic transformation produces better results than other popular transformations. Therefore, the logarithmic transformation is recommended as the primary transformation method for preprocessing regression related data.