Real-Time Data Generation and Anomaly Detection for Security User Profiles

Mostly, security user profiles are being generated from real-time sources of users’ data. User profiles generation process involves complicated and comprehensive data analysis mechanisms. Machine learning is, and become the most popular, technique among the researchers and the practitioners for this real-time data analysis. The scope of this paper is twofold: 1) to implement the anomaly detection module for real-time source of users’ data, and 2) to build real-time data generation engine to train and test this model and to provide the researchers an alternative or simulated methods for real-time data source. The data generation engine is powered by the Synthetic Data Vault (SDV) python library and is relaying on both the historical (past) and real-time (fresh) data. The purpose of using historical data generation is to increase the accuracy of anomaly detection model by expanding the user’s activity which will give more insights of the user behavior. The data generation model simulates the real-world data environments to animate the realtime based analytic systems. Anomaly detection model, in other words, is a real-time flagging system utilizing Kafka platform to generate security user profile. Therefore, the security user profiles are generated based on integrated and collaborated tools to enhance the performance of knowledge-based user authentication systems. Finally, this work generates new knowledge which will allow the researchers to implement and train various machine learning techniques with real-time data generation engine.