Challenges in High Performance Big Data Frameworks

Nowadays, we live in a society with billions of devices that are interconnected and interact together to improve the quality of our lives. The management and processing of information and knowledge have by now become our main resources, and the fundamental factors of economic and social development, and it is achieved through Big Data Frameworks (BDFs). The amount of such data is becoming larger every day, and this calls for scalable and reliable BDFs, that can process such data also with real-time requirements. For example, the data collected by an autonomous car should be processed, combined, and interpreted as fast as possible in order to guarantee a safe interaction with the surrounding environment, and of the passengers. This paper analyses the main limitations of current BDFs while providing some key challenges for increasing their flexibility. In particular, we focus on performance aspects, envisioning adaptation as a viable way to automate and improve performance in Big Data Applications.