Performance comparision of Hadoop and spark engine

Data has been growing at an exponential rate in the recent era. This data has to be processed and analyzed carefully to get new insights. So there is a need for a platform that can perform efficient data processing, which leads to the building up of platforms like Hadoop and Spark. Hadoop Map Reduce model helped to process it in a distributed fashion with computation being done at multiple nodes. In spite of the remarkable processing power it still, has some shortcomings. Spark, a computation engine can solve some of the problems involving Iterative/Machine learning queries by caching some of the results from previous queries. Although Spark is faster than Hadoop in most of the iterative applications it is constrained by memory requirements. This paper briefly discusses Spark and Hadoop architecture, their theoretical differences and the comparison of their performance.