Literature Adventures with MEM in LIWC

Background: A dissertation literature review can require up to six months of effort for a chapter's worth of either chronological or thematic history of the ideas and arguments for a given topic. With the fast propagation of engineering education (EnEd) research, there is a correlated need to quickly evaluate relevant research, especially during the initial stages of project design. A way to speed up the process of reviewing relevant literature is to use computational tools. One such example is Linguistic Inquiry and Word Count (LIWC, pronounced “Luke”) and its Meaning Extraction Method (MEM) topic modeling option as a pathway to a speedier literature review over traditional methods (e.g., by hand or spreadsheet). Purpose: We used LIWC to conduct an enhanced literature review as a part of an ongoing study on middle school outreach, engineering identity (eID), and persistence for women in engineering. Here, we highlight the findings gleaned from MEM, the “bottom-up” topic modeling procedure to find common themes within diverse types of EnEd text, also weighing the importance of each search term within the paper set while creating an author reliance metric. Scope/Method: LIWC quickly gathered data on a set of twenty-seven papers found via systematic search methods, using both “top-down” methods with user-defined and built-in dictionaries as well as the “bottom-up” MEM analysis. MEM first uses a find-and-replace lemmatizer, then it scans for the number of times those simple forms appear (also counting the number of documents) and produces a document-term matrix. We analyzed the matrix for the top-50 most frequently used vocabulary within all papers and top-10 terms within a majority of the papers and compared the results to a priori eID and persistence facets. Results: In this early-stage work, we learned that computational text analysis could indeed speed up the process of a literature review. LIWC MEM analysis confirmed the most important topics to the body of texts analyzed and was able to determine that the recurring verbiage in these texts indeed applied to the topic under study while swiftly highlighting the recurrence of the original search terms. Conclusions: LIWC allowed our team to focus quickly on the most salient themes in the literature. Still, we reaffirm that we had to read the papers found to capture the essential insights. The power of the MEM portion of LIWC might be the future for understanding the language of EnEd research in a quantitative manner and could allow faster inter-rater consistency if also used to analyze transcripts.