Determination of the Optimal Number of Topics in the LDA Model When Working with Large Arrays of Text Data

The article presents the issues of thematic modeling. The existing approaches for determining the optimal number of topics when working with text data are considered. Calculated metrics such as coherence score, perplexity. There are many widely used methods for determining topics when working with text data. But most of the methods are developed based on the Latent Dirichlet Allocation (LDA). One of the main disadvantages of the probabilistic LDA model is the parameter
$\theta$
, which the researcher must determine in advance, which is often impossible to know a priori. One of the main adopted strategies is the calculation of a series of LDA models with a different number of topics in a certain quantitative range, followed by the calculation of the quality metrics of the resulting topics.