Big Data: Current Challenges and Future Scope

Big Data encompasses huge amounts of raw material which influence multitude of research fields as well as different industries performance such as business, marketing, social network analysis, educational systems, healthcare, IoT, meteorology, fraud detection. It aimed to uncover hidden trends and has prompted a development from a model-driven perspective to a data-driven approach. Among numerous properties of Big Data, datasets of Big Data are identified primary as 3Vs attributes which have high variety, velocity and volume. These provide an invaluable insight and assist in making precise decisions. Analyzing this information and outlining the outcome into helpful data is the method for extricating an incentive from these enormous volumes of datasets. Nevertheless, Big Data containing unique features that cannot be handled and processed using the conventional methods. This has presented a significant challenge to the industry. This research paper presents a general outline of the characteristics of Big Data as well as expounds on the present challenges and limitations in this area. It further discusses the future scope in particular the future direction for Big Data research.

I. Introduction

A lot of research is ongoing in the area of Big Data by computer scientists from the industrial as well as the academic fields [1]. Large and diversified data are being gathered and stored on a daily basis. These data are derived from various sources including devices, sensors, networks, videos/audios, log files, applications, social media, websites, etc. The significance of the Big Data does not rely on the amount of data that is stored but whether it can be Analyzed for finding solutions to cost and time reductions. The solutions should lead to new product developments, as well as making intelligent decisions[2]. Big Data tools including Spark, Hadoop, etc. and techniques such as Data Mining, Machine Learning, etc. are being used to extract information and for analysis of Big Data [3]. Deriving precise results from these data analysis has an important function in making accurate and beneficial decision in various fields (e.g., business, marketing, social network analysis, educational systems, healthcare, IoT, meteorology, fraud detection, etc.)