A Review of Natural Language Processing Techniques for Sentiment Analysis using Pre-trained Models

Research and industry are becoming highly interested in automatically analyzing the opinion of general public from social networks with respect to a particular subject. Extracting the polarity from these data is always remaining as a significant bottleneck. Pre-trained models built on deep learning architecture can achieve this task in an effective manner by using transfer learning approach. Since it is difficult to develop a model from scratch, due to time constraints or computational limits, pre-trained models with vast potential and possibilities were introduced. They provide a benchmark to either improve the existing model or test the developed model against it. This paper discusses about various word embedding methods used for sentiment analysis followed by an overview on state-of-the-art pre-trained models used for natural language processing, which is commonly used in the process of sentiment analysis. Experimental results of two state-of-the-art pre-trained models are also analyzed.

I. Introduction

Sentiment analysis is widely used for analyzing opinions of people towards various entities such as products, services, topics etc. and there by evaluating customer satisfaction. It is also named as opinion mining which consists of large problem space. Sentiment analysis includes Natural Language Processing (NLP), text analysis and computational linguistics for identifying and extracting subjective information in source materials – for example, checking whether a review is positive or negative. Analysis of sentiments, extraction of opinion, mining sentiments and opinion, subjectivity analysis, emotion analysis, review mining etc. is different names used for slightly different tasks.