An Iterative Clustering Approach for Tracking Server Logs for Monitoring SCADA EMS/DMS

Due to continuity of operation, software upgrades and patches may need to be incorporated without adequate testing. Tracking system log messages allows us to compare a system’s behavior over periods of time. We present an iterative clustering approach for weekly system logs that are maintained by servers in SCADA EMS/DMS systems. The main goal is to monitor behavior from one week to the next, for the purpose of summary reporting and diagnostics. The algorithm identifies clusters iteratively by reducing the data-set to a remaining set of outliers at each iteration. Details of the identified clusters are retained for further exploration and analytics. A set of tracking scores is obtained as a concise representation of weekly system behavior. The tracking scores are based on different metrics. We demonstrate the application of our algorithms using two real-life datasets from SCADA EMS/DMS. We also extend basic clustering algorithm through application of template matching rules to assign feature labels to each line of an input log, thereby preparing the datasets for machine learning applications. The algorithms presented are proposed as a means for monitoring weekly operations as well as studying the effect of long term changes caused by patched and upgraded software.
^1
1
This research was funded by a grant from the Energy Market Authority of Singapore (EMA) through Grant RGEMA1901, under the Energy Programme National Cybersecurity R&D Grant call 2018 (NRF2018-NCR003)