An Application of Automated Machine Learning Within a Data Farming Process

Data farming is a simulation-based methodology used within the defense community to analyze complex systems and provide insights to decision makers. It can produce very large, multi-dimensional data sets that require sophisticated analysis tools, such as metamodeling. Advances in explainable artificial intelligence have expanded the types of metamodels that can be considered; however, constructing a well-fitting machine learning metamodel involves many tasks that can become time consuming for an analyst. Automated machine learning (autoML) can save an analyst time by automating metamodel training, tuning and testing. Using outputs of an agent-based simulation of a military ground-based air defense scenario, we compared the performance of metamodels trained using autoML and different experimental designs. We found that autoML can reasonably automate the construction of metamodels and adds robustness to the analysis by considering multiple types of metamodels; however, the type and size of experimental design can significantly impact metamodel performance.