Exploring Transformer Ensemble Approach to Classify Cyberbullying Text for the Low-resource Bengali Language

Cyberbullying is one of the recent alarming issues for society, where an individual or a community is attacked or dispraised by an unlawful message or text via electronic communication devices with the help of the Internet. The consequence of this bullying text humiliates the people, and sometimes it initiates religious and racial chaos. In the worst cases, victims attempt suicidal damage. Although scholars contribute more to resolving cyberbullying issues in the English language, a limited contribution to the Bengali text data is identified because of insufficient resource. Consequently, this paper focuses on the Bengali language, an example of a low-resource language for recognizing five classes of cyberbullying text: Not Bully, Troll, Sexual, Religious, and Threat. In this regard, a new ensemble approach is introduced from five state-of-the-art transformer models: multilingual Bidirectional Encoder Representations from Transformers, XLM-RoBERTa, DistilmBERT, BanglaBERT, and Bangla-BERT-Base. This proposed approach surpasses the performance of the existing classification systems by obtaining 87.61% accuracy and 87.59% F1-score.