Enhancing Agile Effort Estimation: An NLP Approach for Software Requirements Analysis

The accurate estimation of project effort is a crucial objective in software development processes. Given that the communication of task descriptions typically occurs in natural language texts, Natural Language Processing (NLP) methods within machine learning have the potential to provide rapid and effective ways. The objective of this study was to enhance the accuracy of the effort estimation classification task for software requirements by proposing an NLP-based software requirements effort estimation model for agile software development processes. A semi-supervised noise filtering mechanism based on k-means clustering with tf-idf embeddings was implemented and evaluated. The software requirement documents were represented by FastText embeddings, and then a fast text classifier was used to predict the expected effort of a given requirement text. The effectiveness of the implemented model was evaluated and it was revealed that the application of noise filtering has improved the performance with an accuracy of 96.8%.