Power and energy characteristics of MapReduce data movements

Worldwide data centers consume about 300 billion kWh of energy per year, which accounts for 2% of total electricity use. As MapReduce becomes the mainstream paradigm for data-intensive computing in data centers, optimizing MapReduce energy efficiency can greatly mitigate energy requirements and reduce energy bills. Numerous studies have attempted to improve MapReduce energy efficiency, but few have approached this problem from understanding and reducing the energy impact of data movements. As data movements are often performance and energy bottlenecks, we propose a data movement centric approach and present an analysis framework with methods and metrics for evaluating costly built-in MapReduce data movements. Our experimental investigation leverages the fine-grained performance and power profiling framework eTune and reveals unique system-level and component-level energy characteristics of data movements. It also shows the scalability of energy efficiency with MapReduce workload and system parameters. These energy characteristics can be exploited in system design and resource allocation to improve data-intensive computing energy efficiency.