A dirty data recognition method for machinery condition monitoring in big data era

Condition monitoring of machinery has entered the big data era, while the existence of dirty data reduces the quality of the whole data. In order to recognize the dirty data included in machinery monitoring data, a new method is proposed in this paper. First, a feature named sampled power index (SPI) is designed to transform the dirty data recognition issue into the outlier recognition. Then the windowing technique, the difference operation and the logarithm transform are introduced to reduce the feature tendency and the feature volatility. Next, auto regression-generalized autoregressive conditional heteroskedasticity (AR-GARCH) model is applied to regress the feature series and produce the crippled local means and local volatilities. Finally, the features are normalized and the 3σ criterion is applied to recognize the dirty data. The performance and the feasibility of this proposed method are evaluated by a simulation and an experiment. The results validate the effectiveness of the proposed method.

I. Introduction

Modern machinery has become more integrated and complicated than ever, and inevitably, the condition monitoring (CM) system is applied to monitor machinery operating conditions. The condition monitoring of machinery tends to require rapidly increasing monitoring sensor numbers, sharply growing sampling frequencies and persistently extending monitoring durations, which has brought dramatic acceleration to the data accumulation process and led the machinery condition monitoring into the big data era [1]. The accumulated massive data are generally complicated in structure and low in information density. Namely, available data and dirty data mix up and interplay in complex manners [2]. Recognizing and eliminating the dirty data in datasets, which are caused by unknown machinery conditions, recording mistakes, etc. and do not conform an expected behavior, from the available data would help improve the data quality and promote the data analysis process. Thus it is worthwhile for researchers to investigate the dirty data recognition methods.