How to Process, Model and Apply Heterogeneous Data Obtained from the IoT

Now, IoT systems generate massive amounts of data in diverse formats and shapes. Integrating information flows is essential for scientific, technical, and business practices. However, processing and using data in different formats and forms requires appropriate competencies, making it difficult to organize the entire process to achieve sustainable and reproducible results. Heterogeneity is one of the main characteristics of the data obtained from IoT systems, and this is the main reason for the problems arising in many cases of data analysis and application. This paper presents a systematized end-to-end methodology for processing, modelling and implementing heterogeneous data obtained from IoT devices. A general conceptual scheme of its development is offered. The theoretical-methodological framework is described and explained in detail as phases, grouped into four main stages. The pipelines are built by defining the stages, which show the necessary actions for each step. A systematized end-to-end methodology can be applied in different areas where IoT systems are used. The proposed end-to-end workflow is realized in the AWS cloud Serverless environment as a pipeline.