Comparative Analysis of the Use of Pre-Trained Models to Profile Authors’ Ages and Genders

Many researchers have contributed to the field of authorship profiling and offered a variety of Artificial Intelligence methods, including machine and deep learning. The advent and application of transformer-based pre-trained models such as BERT and other successive models enabled researchers to improve the accuracy of applied models in different natural language processing domains. A crucial domain that attracted our attention is authorship profiling, wherein anonymous authors’ demographic information (e.g., age and gender) is determined. In this paper, we present a comparative study wherein we examine which of the selected pre-trained models perform the best when determining authors’ ages and genders; we employ the BERT, RoBERTa, and BERTweet pre-trained models and experiment with different hyperparameter values to observe the effect of each associated variable. We find that the selection of certain hyperparameters allow the models to successfully converge; we conclude that with specific hyperparameter values, the BERT model yielded the highest result for the age- and gender-profiling tasks.