Applying Fuzzy Linear Regression to Understand Metacognitive Judgments in a Human-in-the-Loop Simulation Environment

In this paper, a new approach to evaluate metacognitive activities is investigated using fuzzy linear regression analysis. Metacognition shows a broad picture of learning competencies that significantly influences learning processes such as confidence judgment and control of learning. However, it is hard to detect changes in metacognitive judgments because there is no direct way to evaluate metacognition while individuals are learning a new task. We investigated the internal relationship between an individual's metacognitive judgments and task performance. Participants performed a radar monitoring task by playing the role of an antiair warfare coordinator in a human-in-the-loop simulation. In order to measure task performance, participants were given a situation awareness (SA) probe. To measure their metacognitive judgments, we administered a retrospective confidence judgments (RCJ) probe. A fuzzy linear regression model was used to analyze the relationship between RCJ and SA. There were three groups in this experiment. The first group (SA + RCJ feedback) viewed their SA performance with the correct answers to all SA questionnaires and triangular graphs of both SA confidence and SA scores together. The second group (SA feedback) only watched their SA performance with the correct answers to all SA questionnaires. The third group was the control group, and it did not observe any feedback. The results showed that the SA + RCJ feedback screen could significantly influence the participants' mental state from overconfidence to underconfidence as well as SA accuracy. Using the outcomes of the experiment, we modeled mental state change in metacognitive judgments using fuzzy linear regression.