Fine-Grained Elasticity for Big Data Stream Processing of IoT Applications

In an era when technology is developing rapidly like nowadays, there will be a large amount of data generated from technological devices, especially Internet of Things (IoT) devices, and so the problem raised is how to make effective use of that amount of information. To do this, it is necessary to build a system that can handle big data with the ability to scale resources depending on the amount of information received, which is able to bring both work and economic efficiency as well as avoid wasting resources. IoT smart homes are becoming more and more popular and these houses are equipped with many IoT devices to improve people's quality of life. Every second these smart devices generate a lot of data, so a real-time processing system will be needed to monitor and control them to suit different needs. Ensuring optimal resource usage in this context may require fine-grained elasticity, a characteristic of Cloud computing. Therefore, this paper focuses on solving the big data processing and the implementation of fine-grained elasticity on the Apache Storm platform as a solution to improve the efficiency of resource consumption during the operation of the IoT applications. We also tested and evaluated the pros and cons of the proposed solution when applied to Storm worker nodes. Experiments are based on the DEBS 2014 practical problem to find an appropriate solution to process energy usage data sent over MQTT protocol from large number of smart plugs placed at 40 smart buildings in a German city.