Regularized Phrase-Based Topic Model for Automatic Question Classification With Domain-Agnostic Class Labels

Classification of questions according to domain-agnostic class labels relies on a suitable feature extraction process. We propose the use of phrases that is more effective than using words to represent questions. The proposed phrase-based topic modeling technique employs asymmetric priors that are scaled with a new C-value for nested regular expressions. In addition, to suppress high-frequency words in phrases, we deploy term weightages computed using the modified distinguishing feature selector. The proposed approach also incorporates a new topic regularization mechanism to facilitate efficient mapping of questions to class labels. We validate the performance of our proposed model via four datasets across different domain-agnostic class labels comprising question types, reasoning capabilities, and cognitive complexities. Results obtained highlight that the proposed technique outperforms existing methods in terms of macro-average F1 score.

I. Introduction

Automatic question classification (AQC) has been proposed for several applications and is achieved by defining domain-specific or domain-agnostic class labels. Domain-specific AQC is commonly applied in factoid question answering [1] with class labels being defined by subject matter (e.g., science, arts/humanities, business/finance) [2]–[4]. In contrast, domain-agnostic AQC is applied in information query or dialogue interactions in which the class labels may comprise question types (e.g., true/false, procedural, multi-part) [5] or reasoning capabilities (e.g., multi-hop, comparison, algebraic) [6], [7]. To enhance the effectiveness of deliberate practice [8], assessment questions are classified into their respective cognitive complexities (e.g., comprehension, synthesis, evaluation) for instructors to determine learners’ proficiency levels [9], [10].