Bot or Human? Detection of DeepFake Text with Semantic, Emoji, Sentiment and Linguistic Features

Detecting machine-generated text (MGT), also known as Deepfake text, has become increasingly important in Artificial Intelligence (AI) age and social media platforms. With the proliferation of MGT and the potential consequences of its dissemination, there is a pressing need to develop effective methods for distinguishing between MGT and human-written text (HWT). Our research aim has two-fold: firstly, to examine the inherent differences between MGT and HWT on Twitter, and secondly, to develop a classifier specifically designed for MGT detection on the platform. This classifier utilizes contextualized text embeddings as its foundation while considering additional linguistic features, sentiment features, and emoji embeddings. Our experimental results demonstrate that incorporating additional features enhances the model's ability to detect MGT. Combining fine-tuned BERT embeddings with emoji and linguistic features using a multi-layer perceptron classifier achieves the highest accuracy rate of 88.3%. Our analysis reveals distinct characteristics of MGT compared to HWT, including differences in engagement behavior, linguistic patterns, named entities, sentiment expressions, and text perplexity. Our research contributes to the field of MGT detection by offering a comprehensive approach that combines semantic text embeddings with supplementary features. The proposed model provides a significant step forward in addressing the challenge of Deepfake text detection.