A Method for Measurement Data Modeling and High-Dimensional Outlier Detection Based on Large Dimensional Matrix

In the research of Large Scale Networked Control Systems, the real-time analysis of the big data from the measurement of thousands of light, machinery, electrical components and sensors is the inevitable requirements of Networked Control Systems. How to analyze outliers from massive and high-speed measurement data among thousands of nodes in the whole network, and how to find outliers from mass data is an important research topic of scientific big data mining. The Curse of Dimensionality makes many existing methods of outlier detection no longer valid for high-dimensional dataset. In this paper, we propose a local outlier detection factor based on weighted subspace and further propose an effective outlier detection method for high-dimensional data. The method firstly recognizes the local neighbor-space of each data point according to its KNN, and then calculate the sparse factor and subspace weighted vector, which can effectively reflect the local outlier factor and outlier-correlated subspace. After that, an effective outlier detection algorithm for high-dimensional dataset is proposed. we conduct extensive experiments to validate the correctness and evaluate the effectiveness of the proposed algorithm on the real-world dataset.