Classifying AI-generated summaries And Human Summaries Based on Statistical Features

In an age where artificial intelligence knows no bounds, it's crucial to know if the textual content is reliable. But, the task of identifying AI-generated content within vast volumes of textual data is a big challenge. The existing studies in feature-based classification only explored prompt-based text responses. This paper explores methods to identify AI-generated summaries using feature-based machine-learning techniques. This study uses the BBC News Summary dataset. The summaries for the dataset are then generated using three of the top-performing summarisation models. Different statistical features like Zipf's Law Score, Flesch Reading Ease Score, and the Gunning Fog Index are used for extracting features for the classification model. The aim is to differentiate AI-generated summaries from human-written summaries. The main part of the study involves extracting the statistical features from the summarized texts, which are then classified using different classification models. Different models like Support Vector Machine (SVM), Random Forest, Decision Tree, and Logistic Regression models are used in the paper. Grid Search is also used to fine-tune SVM for the best results. The right model depends on what the need is. Whether it's accuracy, F1 score, or a mix of both, there are different options to lead you to the truth. The feature-based approach in this paper helps in more explainable classification and can compare how statistical text features are different for human-written summaries and generated summaries.