An End-to-End Framework for Provisioning-Based Resource and Application Management

Resources in distributed infrastructure such as the grid are typically autonomously managed and shared across a distributed set of end users. These characteristics result in a fundamental conflict: resource providers optimize for throughput and utilization which coupled with a stochastic multiuser workload results in nondeterministic best effort service for any one application. This conflicts with the user who wants to optimize end-to-end application performance but is constrained by the best effort service offering. Resource reservations can be used to obtain more predictable application behaviors but they are generally not allowed due to perceived impact on the other users and overall resource utilization. In this paper, we examine two strategies for integrating reservations within the resource management fabric that address these concerns by either minimizing the adverse impact of a reservation on the other users or enabling a resource provider to recoup losses through a differentiated pricing mechanism. Correspondingly, we also present algorithms for optimizing the application performance when resources provide automated reservations using the previously developed strategies. These algorithms use a cost based model to identify the set of reservations to be made for the application in order to optimize performance while minimizing the cost for the reservations. The cost based model allows the users to do a tradeoff between the application performance and resulting resource costs. Using trace-based simulations and task graph structured applications, we compare the application performance and resource cost when it is executed using reservations to that when only best effort service is available. We show the approach incorporating reservations can provide superior performance for the application at a price that the user can predetermine. Also, the benefits of using the reservation-based approach become more pronounced when the resources are under high utilization and/or the applications have significant resource requirements.

I. Introduction

Shared distributed infrastructures such as the Teragrid [1], the Open Science Grid [2], and others can provide the software and hardware resources for cross-organizational collaborative work. These types of collaborations have proven to be of value across a wide range of scientific fields such as astronomy [3], high energy physics [4], earthquake science [5], etc. Such collaborations, also called virtual organizations (VO) [6] require resources from multiple providers for executing large-scale scientific applications. Furthermore, emerging classes of deadline-driven scientific applications such as severe weather modeling [7] require simultaneous access to multiple resources and predictable completion times.