English to Kannada Translation Using BERT Model

Translation of content on the internet from one language to the other is essential for conveying information to the people around the world. This can be text, audio or video-based information. Today, the BERT model which stands for Bidirectional Encoder Representations from Transformers helps computers understand the meaning of ambiguous language in text by using surrounding text to establish context. In this paper, an English to Kannada translation model using the pre-trained uncased BERT framework has been presented. Low-resource language like Kannada requires a vocabulary creation and an appropriate preprocessing. WordPiece and SentencePiece tokenizers have been used for appropriately clean and prepare the data for model building. Uncased pre-trained BERT model is used as an encoder for model building and the FairSeq algorithm is used for decoding. The model is trained and tested on Samanantar dataset consisting of 6 lakhs of records. The proposed custom model has outperformed for about 29% compared to multilingual BERT model and it also outperformed ordinary transformer model by 50%. These findings have proved that custom language models perform better in case of Indic languages, especially the Dravidian languages like Kannada.