Improving Human Situation Awareness in AI-Advised Decision Making

Human-autonomy teams aim to leverage the different strengths of humans and autonomous systems respectively to exceed the individual capabilities of each through collaboration. Highly effective human teams develop and utilize a shared mental model (SMM): a synchronized understanding of the external world and of the tasks, responsibilities, capabilities, and limits of each team member. Recent works assert that the same should apply to human-autonomy teams; however, contemporary AI commonly consists of “black box” systems, whose internal processes can not easily be viewed or interpreted. Users can easily develop inaccurate mental models of such systems, impeding SMM development and thus team performance.We seek ways to support the human’s side of Human-AI SMMs in the context of AI-advised Decision Making, a form of teaming in which an AI suggests a solution to a human operator, who is responsible for the final decision. This work focuses on improving shared situation awareness by providing more context to the AI’s internal processing. We hypothesize that this will lead the human to a more accurate mental model of the task and the AI, which in turn will improve team performance. We manipulate the human’s situation awareness of the task environment and measure effects on the shared mental model. A between-subjects, randomized experiment is conducted in which participants in 6 treatment groups of varying amounts of contextual information (as a proxy for situation awareness) complete a task with an AI teammate. We find that improving shared situation awareness of decision points improves the human’s overall performance, as well as their understanding of their AI teammate, without directly explaining the AI’s internal mechanisms. Additionally, we find that increasing the human’s situation awareness of task environment and AI teammate reduces over-reliance on the automated teammate.