A Machine Automated Big Data Modular Framework for Finding Network Security Vulnerabilities

Because of the availability of large-to-big data, conventional data processing applications are at their breaking point. The sheer amount of traffic on the network makes it difficult to spot quickly and stop security intrusions. The fastest cybersecurity detection system should handle voluminously to discover malicious traffic immediately. This paper is using Spark, a special-purpose data processing infrastructure that handles enormous amounts of network content. The article is proposing a thriving feature-based feature-selection technique that is often advocated. Here, two widely recognized feature selection algorithms have been used: Pearson Correlation Feature Selection and Fisher Exact Test Feature Selection and other five famous FS schemes Probit, Import Vector Machine, Bernoulli Naive Bayes, K-nearest Neighbours, and Linear Discrimination Analysis. The suggested approach has been evaluated using the CIC-NSL-KDD dataset. The analytical findings demonstrate that it outperforms and is more stable than conventional approaches related to training time, extrapolative time, exactness, sensitiveness, and last but not least is explicitness.