Multi-Objective Deep Reinforcement Learning for Function Offloading in Serverless Edge Computing

Function offloading problems play a crucial role in optimizing the performance of applications in serverless edge computing (SEC). Existing research has extensively explored function offloading strategies based on optimizing a single objective. However, a significant challenge arises when users expect to optimize multiple objectives according to the relative importance of these objectives. This challenge becomes particularly pronounced when the relative importance of the objectives dynamically shifts. Consequently, there is an urgent need for research into multi-objective function offloading methods. In this paper, we redefine the SEC function offloading problem as a dynamic multi-objective optimization issue and propose a novel approach based on Multi-objective Reinforcement Learning (MORL) called MOSEC. MOSEC can coordinately optimize three objectives, i.e., application completion time, User Device (UD) energy consumption, and user cost. To reduce the impact of extrapolation errors, MOSEC integrates a Near-on Experience Replay (NER) strategy during the model training. Furthermore, MOSEC adopts our proposed Earliest First (EF) scheme to maintain the policies learned previously, which can efficiently mitigate the catastrophic policy forgetting problem. Extensive experiments conducted on various generated applications demonstrate the superiority of MOSEC over state-of-the-art multi-objective optimization algorithms.

I. Introduction

Serverless computing is an emerging cloud computing framework in which applications are constructed from fine-grained functions, known as Function as a Service (FaaS) [1], [2], [3]. Within this model, developers focus solely on coding and deploying functional components, whereas managing the underlying hardware and servers falls upon cloud service providers. Concurrently, resource consumption can dynamically scale with the demands of applications. As such, users incur costs for the resources they actually consume rather than for continuous server operation. Prominent platforms for serverless computing include AWS Lambda [4], Google Cloud Functions [5], and Azure Functions [6].