Analysis of Corpus Development for Urdu Language

A corpus is a collection of machine-readable linguistic data, gathered as an authenticated written text or a transcription from a recorded speech, with features examined using various methodologies. In the field of Natural Language Processing (NLP), corpus plays a critical function. Urdu is a major language in South Asia, as well as the national language of Pakistan. A lot of NLP related research has been made for various Western languages but Eastern languages, particularly South Asian languages remained deprived of advanced level NLP applications mainly due to unavailability of language and linguistic resources. Same is true for Urdu corpora, which is not widely available to the researchers. The current study focuses on the collection and development of a corpora, consisting of 100k (100,000) Urdu text words and its analysis using several methodologies. The corpus is gathered from 79 different resources such as Urdu articles or columns published in Pakistani newspapers and journals between 2015 and 2021. Each article is over 1000 words long. This corpus is released and is made freely available to the researchers to develop NLP applications for Urdu language.