Electricity Consumption Data Analysis Using Spark Framework

Smart meter generates streaming electricity consumption data at specific time intervals. Analysis of this data will be more helpful to identify customers of similar consumption patterns. Further, prediction of future electricity consumption helps in the utility for the production plan. However, smart meter generates huge data at high velocity. Hence, it needs to be properly analyzed to derive useful information. Analysis techniques such as clustering and regression can be utilized to analyze this large volume of data to derive meaningful pattern and to predict the future consumption respectively. Existing research works have utilized k-means and multi linear regression method for analyzing smart meter data using hadoop framework. However, these works did not handle the streaming electricity consumption data. Hence, the objective of this project is to propose a big data application framework named Prediction of Electricity Consumption (PEC) system to analyze the streaming data using spark frame- work.