Using Reward Shaping to Train Cognitive-Based Control Policies for Intelligent Tutoring Systems

Intelligent tutoring systems are used to train humans by personalizing education systems. For conventional learning contexts such as mathematics or computer programming, agents in intelligent tutoring systems have been designed to respond to humans based on cognitive feedback. However, the same cannot be said for psychomotor learning contexts. In this paper, we design and validate several model-based cognitive control policies that determine when to provide learners with automation assistance in a psychomotor task. The shaping of rewards used to train these policies is motivated by the important role of learners' self-confidence while learning. The trained policies are implemented empirically in a user study utilizing a quadrotor landing simulator, and learners' performance and self-confidence during the task are compared. The results show that the cognitive control objective of calibrating learners' self-confidence to their performance leads to significantly better task performance than is achieved when the decision to provide automation assistance is driven solely by users' performance. This motivates the need for continuing research in modeling of human cognitive factors and the design of appropriate control objectives grounded in literature on human cognition.