Implementation of Big Data Analytics for Machine Learning Model Using Hadoop and Spark Environment on Resizing Iris Dataset

The concept of Big Data to refer to huge volumes of data and attributes, but data samples through the use of a diverse set of features gathered from various sources. A significant amount of time is spent constructing a pre-processing workflow and an analysis process that make possible impactful for machine learning. Big Data analytics is being driven by the need to process Machine Learning data, actual real-time processing, and graphics processing. Hadoop and Spark, both accessible data warehousing frameworks that allow for the distribution and computation of massive datasets across several clusters of computer nodes, are the most efficient prospects for Big Data analysis in a distributed setting. To test the ability of these Big Data Tools, this research use Iris dataset as experimental data which is resized to a larger file. Multinomial Naive Bayes algorithm was employed to create a classification model for Iris flowers using Spark Machine Learning Library. The experimental result reported that there is a difference in accuracy and execution time during testing machine learning performance in Hadoop. The experiment given the best performance used Iris dataset is resized to 148 MB consisting of 5,184,000 samples, the model accuracy reached 95.32% with an execution time of 1 minute 4 seconds. The increase in the number of samples in the dataset is also positively correlated with increasing execution time. However, execution time is relatively cheap in the Hadoop Environment.