Big Data Analytics in Real Time - Technical Challenges and its Solutions

In this digital era, enormous quantity of data are generated through various sources like social media, sensors, RFID readers etc along with significant contribution from industries. e.g. generation of transaction data by banking, financial and retail sectors, digital patient and their diagnosis records by healthcare sector, seismic data by oil and gas sector, TV adds and music data by media and entertainment sector and most importantly government digital contents. These data are generated at a fast space in different formats which is primarily used by/for the organization to obtain meaningful information and helps decision maker/taker for time-bound action-oriented strategy. When analysis is performed at a real time, volatile plays a big role which stands for how long data is to be made available or valid and to be retained. So, the sheer volume, velocity, variety, and volatility are signified by Big Data. The challenges are to efficiently store, process, visualize and then analyze in real time. The focus of this paper is to present the open-stack solution for analyzing massive amount of data and to present the real-time information about trends, patterns and also to generates alert in case of business emergency. In addition, the paper also presents hardware topology with open-stack solution.