Study of Various Techniques for the Classification of Hateful Memes

An Internet meme is a catchphrase, concept, or piece of media that is spread by means of the Internet, regularly through online media platforms and particularly for comical purposes. Detection of Online Hateful speech in the form of memes presents as a major challenge due to its multimodal nature. This study explores various methods for the classification of hateful memes on multimodal data using several pre-trained models like VGG19 and Xception in combination with machine learning models like Support Vector Machine and Naïve Bayes. We have experimented several techniques where one of them being integrated stacked model technique for the image modality. In our multimodal experiment approach, we have used early fusion technique and GloVe Embedding for which our model obtained 0.584 as the highest f1-score in a multimodal experiment consisting of an LSTM and VGG16 model.