Part-of-speech Tagging and Named Entity Recognition Using Improved Hidden Markov Model and Bloom Filter

Natural Language Processing (NLP) makes interaction with computer easier by making use of human language understanding. Part-of-speech (POS) tagging is first task in every NLP application. The complexity of computing POS tag lies in the number of computational steps an algorithm uses for determining POS tag for a given sentence. If the number of comparisons to do POS tagging can be reduced, then it will benefit all NLP applications. Identifying the entity i.e. Named Entity Recognition (NER) is the basic operation in NLP. NER is done by the help of POS tags. Research work presented here focus on the number of comparisons made by 4-gram Hidden Markov Model (HMM) for POS tagging and how it can be reduced. In this paper, we present a method to decrease the number of comparisons for POS tagging in 4-gram HMM model. In addition, a method has been put forward for NER using Bloom Filter.