Advanced NLP Based Entity Key Phrase Extraction and Text-Based Similarity Measures in Hadoop Environment

Text Analytics is the process of parsing textual content from massive non-structured sources like different blogs, emails, tweets, HTML files, full-text documents, articles and extracting relevant facts. Text Analytics in big data is a multidisciplinary domain. Information Extraction, named entity extraction, is a subdivision of text analytics. Natural language Processing (NLP) is an extensive area for linguistic analysis of the text by using segmentation, tokenization, POS tagging, and syntactic parsing. Big data along with NLP will overcome major challenges in different domains of analytics. In this paper, we have presented some common and basic approaches like the BOW model, TF-IDF Model, Vector Space Model, and Word2Vec model along with their advantages and limitations and made a comparative analysis over enormous documents for finding accurate content, and semantic relationship between words and resolving ambiguities issues. We will also propose an NLP-based information extraction framework for processing and analysis of text documents in a Hadoop-based big data environment.