Advanced Monitoring Infrastructure System Architecture for Massive Multi-Source Data

Advanced metering infrastructure(AMI) is an important part of the smart grid, mainly consisting of metering terminals for data collection and master systems for data analysis. With the continuous development of smart grid, the number of metering terminals is increasing significantly, and the terminal data to be processed will grow exponentially, putting higher requirements on the performance of AMI system data processing. Meanwhile, the security and reliability of the system cannot be guaranteed due to the lack of monitoring of terminal status and real-time perception of network segment traffic, and the establishment of the system security defense system is still in its initial stage. To meet the demand for real-time processing of massive data, realize the rapid transformation of multi-source, multi-location, and multi-temporal data, and guarantee the safe and stable operation of AMI, this paper proposes an AMI monitoring system architecture for massive multi-source data, which uses deep packet detection technology and clustering methods to filter and statistic the data stream. Combined with the microservice technology to build a security monitoring platform, it solves the problem of rapid extraction, analysis, and storage of massive data, and realizes the monitoring of terminal behavior status and real-time perception of network segment traffic situations.