Optimizing Real- Time Threat Detection with Feature Engineering Strategies for Intelligent NIDS

The escalation of cloud-based network attacks has been observed to be in tandem with the rapid proliferation of cloud-based services worldwide. It has highlighted current network intrusion detection systems' (NIDS) inadequacies in adequately safeguarding against distributed security incidents, particularly within the framework of Internet of Things (loT) networks. The significance of intrusion detection systems in safeguarding computer security cannot be overstated, and considerable research endeavours have been directed toward enhancing NIDS methodologies to counteract network interruptions. Learning-based methods to improve detection accuracy by eliminating correlated, recurring, and irrelevant features are a significant area of emphasis in advancing NIDS. Prior studies have investigated the integration of Machine Learning (ML), Dimensionality Reduction (DR), and Deep Learning (DL) methodologies to enhance the accuracy of classification outcomes on Network Intrusion Detection System (NIDS) datasets. The variability in selected features, design, and kinds of attacks among NIDS datasets requires thoroughly investigating these techniques across diverse datasets. The objective of this article is to furnish an overview of prevailing frameworks through the utilization of benchmark datasets to detect atypical attacks and obtain a comprehensive understanding of the present status of Network Intrusion Detection Systems. The efficacy of current Network Intrusion Detection Systems is assessed by categorizing feature characteristics and applying diverse machine learning methodologies. The study delves into the topic of feature selection techniques as a means of augmenting anomaly detection. Additionally, they investigate the pre-processing of Internet traffic data. Furthermore, the article explores the potential of conceptualizing virtualized real-time Intelligent NIDS using feature engineering techniques. This paper enhances comprehension of NIDS by consolidating varied research findings, thereby furnishing valuable perspectives for creating more resilient and efficient intrusion detection systems.