Use of Shapley Additive Explanations in Interpreting Agent-Based Simulations of Military Operational Scenarios

Military defense modernization initiatives often involve complex systems that must be understood to inform design, planning, implementation and acquisition decisions. To gain a basic understanding of the system and identify key initial parameters, simulation experiments can be used to generate – or farm – data efficiently and effectively over a large parametric space. While machine learning models can be used for post-simulation analysis to identify key parameters, interpretability and their black-box nature can present challenges when the intent is to provide support to decision makers. In this paper, we apply a model-agnostic method for interpreting machine learning predictions, known as SHapley Additive exPlanations (SHAP), to data farmed from an agent-based simulation that models a military operational scenario. The scenario is motivated by a Canadian Army initiative to modernize its intelligence, surveillance, and reconnaissance assets and abstracted to minimize the complexity of the modeled system and validate the findings of SHAP.