Improvised Transformer Network for NER on Low Resource EnglishHindi Code-Mixed Language from Scratch

The use of Code-mixed languages in social media platforms is a common phenomenon, which poses a new set of challenges for understanding Code-mixed languages in the field of Natural Language Processing (NLP). The implementation of state-of-the-art (SOTA) algorithms is difficult due to the scarcity of available resources. In the work, we focus on one of the primary NLP tasks namely Named Entity Recognition (NER) for English-Hindi Code-mixed language. We propose an improvised Transformer network that learns word and character embedding from scratch and beneficial in processing low resource Code-mixed languages. We use the only available Twitter NER corpus and obtained a slight improvement over SOTA. The proposed transformer network is a general model and in the future can be useful for training low resource NLP tasks from scratch.