Impact of Word Embedding Methods on Software Vulnerability Severity Prediction Models

Software vulnerability severity prediction has recently gained popularity in software engineering. To perform this task, a prediction model has to be developed. The model’s performance greatly depends on the feature vectors used to train it. These feature vectors are formed using textual data present on software vulnerability and converting it to numerical form. There are various methods for feature vector representation using word embeddings. Unlike traditional methods such as TF-IDF, word embedding methods using deep learning map the words into vectors, preserving the semantic relationships between words as well as doing automatic feature selection, which holds great importance when dealing with textual data. Therefore, word embedding methods using deep learning showed promising results over traditional TF-IDF. Hence, in this paper, we conducted a controlled experiment to examine the effect of various word embedding methods for feature vector representation on the performance of the prediction model. The models developed in this study used TF-IDF, word2vec, and GloVe, coupled with SVM, LSTM, and Bi-LSTM classification algorithms. The classification performance of different word embedding methods is analyzed on five different vulnerability datasets of Mozilla products. The experiments showed that the best performance is achieved by using the GloVe method with the Bi-LSTM algorithm giving an average AUC value of 0.81, which is an improvement of 12.5% from the traditional method of feature vector representation.