Robustness Evaluation Method for Spacecraft Anomaly Detection Models

The anomaly detection of spacecraft is critically important for the safety of spacecraft, and for a specific spacecraft, the most suitable anomaly detection method must be chosen. This paper proposes an evaluation method for the robustness of anomaly detection models. Seven common evaluation metrics are calculated for the prediction results of the anomaly detection models. These metrics can be viewed as a point in a seven-dimensional space. Experiments are conducted using data with varying levels of noise, and metric points are calculated. Subsequently, multiple sets of metric points are clustered. The clustered metric points yield average coordinate points representing each cluster of metrics. Experiments are also conducted using noise-free data to calculate metric points as baseline points. The distance between the baseline points and other metric points is then computed, where a smaller distance indicates better model robustness. Experimental results demonstrate that this evaluation method is applicable for assessing the robustness of anomaly detection models for spacecraft.

I. Introduction

Since the launch of the first spacecraft by humans in the 1950s, humanity entered the space age. After decades of development, spacecraft have become increasingly sophisticated in their functionality, and spacecraft systems have grown in complexity, with a diversification of platforms. The consequent challenge is how to ensure the safe operation of spacecraft. Timely sensing of a spacecraft's status and the detection of anomalies are crucial methods for ensuring spacecraft safety. Additionally, due to limitations in measurement methods and potential interference in telemetry transmission, the data we collect may contain noise. Therefore, our anomaly detection methods should exhibit robustness.