Big Data-Based Approach to Automated Linguistic Analysis Effectiveness

The authors of the finding develop customizable corpus tool to build corpus of historical and religious texts. Big Data approach to Natural Language Processing and Natural Language Understanding was used to achieve the goal of such corpus data platform developing. Calculation of qualitative and quantitative characteristics, building search queries belong to the most important features of the adaptable text corpus effectiveness. Number of computer-based calculations and amount of processing data have been reduced and parallelized to achieve higher performance on the levels of computational methods and implemented system. The higher level of efficiency as a trade-off between effectiveness and computational time has been achieved by choosing proper parameters of computational methods. Latent-Semantic Analysis is used as one of the core methods for making queries. The methods applied are mostly based on Singular Value Decomposition. Parameters of the decomposition are analyzed and justified. Suggested approach has been verified on the test data available in different languages.