Hyperparameter Tuning and Validation of Neural Network Model for Software Effort Estimation with NCA Based Feature Selection

Effort estimators and Project managers still struggle with estimating the project effort. Early in a project, a high degree of ambiguity and inexperience lead to inaccurate estimates of the amount of work needed. Soft computing techniques such as artificial neural networks (ANNs), fuzzy logic and genetic algorithms are the most preferred software effort estimation approaches. This study compares two different types of ANNs, feed-forward neural networks (FFNN) and cascade-forward neural networks (CFNN) for software effort estimation. The number of neurons in the hidden layer, and activation functions of layers are hyperparameters to be tuned prior to training. This paper performed trial-and-error approach for tuning the hyperparameters of ANN to reach the optimum architecture of ANN. Nonparametric feature selection technique, neighborhood component analysis (NCA) was utilized to improve the prediction accuracy. The use of NCA improves the prediction accuracy of ANN. This study also demonstrates the need for normalization of the data for accurate estimation of response variables. The work is validated with two datasets i.e. coc81 and china datasets using a 2-fold cross-validation approach using 12 performance metrics. The experimental results show the effectiveness of the proposed technique in contrast to the use of ANN without normalization and feature selection.