Evaluation of Topic Models and Information Retrieval Methods in Support of Lessons Learned and Knowledge Management

A lesson learned is the application of previous experiences to improve decision-making and organizational efficiency. The lack of learning from past organizational mistakes and successes has contributed to extended schedules, increased program costs, and excessive program rework. Applying the lessons learned process is critical in meeting program objectives, mitigating risks, and improving organizational effectiveness by avoiding costly errors. This type of learning requires communication between the source of the knowledge and the receiver(s) of the lesson to magnify the benefits of a lesson learned. A significant factor in gathering, maintaining, sharing, and reviewing lessons learned is for an organization to implement a knowledge management system or a lessons learned database. This system/database allows the team members access to all previous lessons learned, which, as a result, allows the organization to learn from past mistakes/successes by applying that knowledge in the present and future. There is limited research on an automatic lessons learned/knowledge management database that removes the need for users to perform manual searching and provides highly relevant results to the users. One significant step in constructing an automatic lessons learned/knowledge management database is the implementation of topic models. Topic models are machine learning algorithms produced to identify the foundational semantic structures of corpora using Bayesian hierarchical modeling. This research evaluates topic models intending to improve relevancy in lessons learned query searches in knowledge management/lessons learned databases. Widely utilized topic models such as the Latent Dirichlet Allocation (LDA) and Latent Semantic Indexing (LSI) are investigated in this paper in addition to the Term Frequency – Inverse Document Frequency (TF-IDF), Hierarchical Dirichlet Process (HDP), and Random Projections (RP) topic models. The TF-IDF topic model outperformed all topic models evaluated in this work.