Lyapunov-Guided Offloading Optimization Based on Soft Actor-Critic for ISAC-Aided Internet of Vehicles

Due to numerous computation-intensive and delay-sensitive tasks in the Internet of Vehicles (IoV), Vehicular Edge Computing (VEC) is increasingly playing a crucial role as a key solution in the IoV. However, how to concurrently enhance communication quality and reduce the cost of latency and energy has emerged as a critical challenge in VEC. To tackle the above problem, we propose a Lyapunov-guided offloading based on the Soft Actor-Critic (SAC) algorithm, named LySAC, to minimize the average cost of the Integrated Sensing and Communications (ISAC) technology-aided IoV, where ISAC technology can effectively improve the communication quality by harnessing high-frequency waveforms to seamlessly integrate communication and sensing functionalities. First, we model the offloading process of ISAC-Aided IoV as an optimization problem of the joint cost of delay and energy with long-term energy consumption and queue stability. Then we formulate the optimization problem as a Lyapunov optimization and utilize the SAC method to find the optimal offloading decisions. Finally, we conduct extensive experiments and the results demonstrate the effectiveness and superiority of the proposed LySAC in minimizing total cost while maintaining queue stability and meeting long-term energy requirements compared with other several baseline schemes.

I. Introduction

Internet of Vehicles, a subset of the broader domain of the Internet of Things (IoT), revolve around the connectivity and communication among vehicles, enabling seamless data transmitted between vehicles, infrastructure, and other devices to improve road safety, traffic efficiency, and overall transportation effectiveness [1]. However, the continuous influx of data, much of which is time-sensitive, manifests as a flood of computational tasks that overwhelm the limited processing capabilities of vehicles, thereby increasing the risks of on-road incidents. This means that the numerous time-sensitive tasks constitute a contradiction with the limitation of computational resources of the Internet of Vehicles (IoV). A promising solution to the contradiction is Vehicular Edge Computing (VEC) [2], [3], which offloads tasks from vehicles to Road-Side Units (RSUs) positioned along the roads, where the RSUs have advanced computational capabilities surpassing those of the vehicles [4].