Mobile Communications, Computing, and Caching Resources Allocation for Diverse Services via Multi-Objective Proximal Policy Optimization

Mobile services are becoming more diverse, making them have different demands on communications, computing, and caching (3C) resources in mobile systems. Unlike the traditional work that considers only one type of service, this paper designs a unified framework to characterize the different kinds of services, and jointly optimizes the 3C resources of the base station (BS) and mobile devices to provide differentiated quality of service (QoS) for diverse services. In the proposed framework, we model the task required by the mobile device to be generated at the BS, the mobile device, or both of them, which means the requested tasks are served through different paths, consuming different bandwidth, computing and caching resources. Since diverse services have different QoS, we formulate a multi-objective programming (MOP) to optimize the allocation of the 3C resources for minimizing the total delay while maximizing the number of executed tasks requested by the mobile devices. We transform the MOP problem as a multi-objective Markov decision process (MO-MDP) and design a multi-objective proximal policy optimization (MO-PPO) algorithm to solve the MO-MDP. The proposed MO-PPO first trains two sub-policies separately for the two objectives, and then combines them to search for Pareto dominating solutions. By alternately perform the separate training and the combination, we can finally obtain a set of Pareto optimal solutions and the corresponding Pareto front. Simulation results show that the proposed MO-PPO outperforms traditional methods in finding a higher-quality set of Pareto optimal solutions and can more appropriately allocate 3C resources to different types of services.

I. Introduction

The past decade has witnessed a significant increase in the data transmission rate of mobile networks following Shannon’s law. Meanwhile, computing and caching resources of the network and mobile devices have experienced tremendous growth, following the Moore’s law. As a result, new mobile services such as virtual reality (VR), online games are emerging [2]. These services have different requirements on the communication, computing, and caching (3C) resources of the mobile network. Specially, the mobile VR demands large communication bandwidth and large caching size, mobile search engine requires large computing resource but only needs low transmission data. Some services consume high 3C resources, like online games, which are called 3C-intensive services.