Explaining Cyberbullying Trait Detection Through High Accuracy Transformer Ensemble

Cyberbullying remains a pernicious concern within social media which can have an outsized impact on teens and children who are spending more and more of their social engagements online. The recent COVID-19 pandemic physically isolated young students during school and other public closures which served to increase usage of social media. These events heighten the need for careful, explainable, and ethical tools for monitoring and identifying cyberbullying with explaining classification to specific traits. Classical machine learning methods which form a large part of the existing domain provide better introspection into how the ML system arrives at its inference. These traditional stochastic based methodologies are hampered by the complex task of feature engineering and generally lag behind newer deep neural network (DNN) architectures in accuracy. The tradeoff is that DNNs are often opaque with respect to explainability of the model, the inference, and the impact of vast training sets toward ethics and bias. In this work, we propose an advanced architecture that incorporates second and third generation transformer-based models to provide focused and highly accurate cyberbullying trait detection and identification allowing for human verification and validation based on these inferences. This architecture uses the IEEE Fine-Grained Cyberbullying Dataset (FGCD) exceeding current SOA.