Latent-Space Analysis for Improved Overhead Imagery Explainability

With the ever-growing quantity of information generated by commercial and intelligence platforms, it is imperative that the community moves toward solutions that provide not just automated outputs, but a more accurate explanation into why automated outputs are predicted. Automated Target Recognition (ATR) solutions that rely on deep learning are being leveraged to derive intelligence automatically, but these networks can produce incorrect answers in some circumstances, reducing user trust. Most analysts cautiously trust these results, but they typically still require inefficient manual verification. To help address this challenge, we introduce an ATR anomaly metric created specifically for GEOINT systems that determines if a newly detected object fits into the distribution of the network’s original training dataset. This approach has been tested on different sets of targets, such as ships, airplanes, and vehicles, spanning multiple publicly available EO and SAR commercial GEOINT datasets. Low anomaly scores generated by this approach help to validate model performance and increase trust in a model’s outputs, while high anomaly scores help identify unique images that were not captured within a model’s training distribution. For instance, some examples of high anomaly scores were found to correspond to mislabeled data used in training, identifying aircraft in pre/post flight configurations and in various stages of disassembly, detecting boats out of water, and identifying aircraft that were viewed through clouds. More importantly, for detections with a low anomaly score, our analysis shows that the results are highly accurate, allowing the analyst to trust the results and focus on harder problems. Additionally, we show a preliminary proof of concept analysis in which we invert this anomaly metric to perform semi-supervised active learning on brand new datasets.

I. Introduction

Overhead Automatic Target Recognition (ATR) is a class of algorithms that find and/or identify known or unknown targets typically within Electro-Optical (EO), Synthetic Aperture Radar (SAR), and other types of imagery products. In the past decade, the task has shifted from being dominated by hand-crafted rule-based algorithms to Deep Learning (DL)-based approaches that make use of Convolutional Neural Networks (CNNs) that learn to extract relevant image features. To achieve optimal ATR performance, a CNN requires a large and accurately labeled dataset that reflects the operational distribution of targets and images. This labeling process is time-consuming and prone to error. As a consequence, noisy and/or out-of-distribution data are often inadvertently introduced into the training dataset for an ATR network, resulting in degraded operational performance.