Comparative Analysis of Term Weighting Methods for Question Classification in Bloom Taxonomy Using Machine Learning Approach

In higher education institutions, examinations are crucial in assessing students' cognitive levels. Educators manually categorize exam questions according to Bloom's taxonomy cognitive domain. Recognizing educators' challenges, previous research has explored automated question classification using machine learning methods. Feature selection, feature extraction, and term weighting are common strategies to enhance classification accuracy. Unsupervised term weighting methods like TF and TF-IDF have been widely used among these. This paper aims to delve into the effectiveness of TF-IDF, Bag-of-Words (BoW), and Binary term weighting techniques. Three distinct classifiers were used to scrutinize these techniques: Support Vector Machine (SVM), Naïve Bayes (NB), and Random Forest (RF). The research seeks to identify the most suitable term weighting technique and classifier for automating question classification based on Bloom's taxonomy. The average accuracies achieved by TF-IDF, BoW, and Binary using SVM classifier were 77%, 78%, and 77% while using the NB classifier, the average accuracies achieved for TF-IDF, BoW, and Binary were 72%, 72%, and 73%, using RF classifier the average accuracies achieved for TF-IDF, BoW, and Binary were 80% 79%, and 81%. Among the classifiers, the Binary method consistently displayed the best precision, recall, and F1-Measure results, indicating its effectiveness in accurately categorizing positive instances. The TF-IDF approach exhibited competitive precision, although its recall was relatively lower, suggesting possible difficulties in capturing many positive instances. Meanwhile, the BoW technique performed moderately, balancing precision and recall. This study compares several variants of unsupervised term weighting, TF-IDF, BoW, and Binary concerning classification accuracy and identifies the optimal unsupervised term weighting.