Machine Learning and Big Data Based IDS System Extensive Survey

Network traffic and logs are monitored by intrusion detection systems (IDS) for abnormal behavior that could indicate a security breach. Traditional IDS techniques require a considerable amount of time when processing big amounts of data. It also complicates the system and diminishes its effectiveness. In addition, the time required to analyze the data renders the system open to attack for an extended period of time before an alarm is generated. IDSs increasingly employ machine learning algorithms to enhance their ability to identify threats in the presence of massive volumes of data. This paper will introduce machine learning and IDS integration for big data. This connection will enhance the IDS system, increase its detection capabilities, and produce more accurate IDS results. A comprehensive survey of IDS that employ machine learning and big data is offered. Comparison is made between several machine learning algorithms used to improve IDS and the feature selection conducted on a specific dataset.