DRL Connects Lyapunov in Delay and Stability Optimization for Offloading Proactive Sensing Tasks of RSUs

The integration of Roadside Units (RSUs) is vital for the development of autonomous driving technologies. Challenges arise from sinking computing capabilities into RSUs and vehicles in the paradigm of Vehicle Edge Computing (VEC), particularly due to heterogeneous computation and communication capacities of network nodes and multiple sources of computing tasks (node-mounted and offloading tasks). These challenges complicate network stability from the perspective of a long-term optimization evolving over time, considering unpredictable task distribution and environmental states. To tackle these challenges, we approach the problem of partial task offloading to minimize task delay while meeting the demand of system stability over time as a dynamic long-term optimization. Utilizing Lyapunov stochastic optimization tools, we successfully decouple the long-term delay minimization and stability constraint, transforming it into a per-slot scheduling problem. Since the per-slot scheduling problem with complicated Lyapunov drift functions can not be solved by numerical optimization at each time step, our solution leverages a proposed deep reinforcement learning algorithm, leading to extensive simulations that demonstrate the superior effectiveness and efficiency of our proposal compared to existing schemes.

I. Introduction

In the development of autonomous driving technologies, Roadside Units (RSUs) are recognized as a critical infrastructure component for their sensitivity to time delay in intensive computations in the Internet of Vehicles (IoVs) [1]. RSUs are typically equipped with various sensors and communication devices, allowing them to sense proactively and process environmental data in real-time. Therefore, RSUs improve the overall efficiency of IoVs by supporting task offloading from autonomous vehicles [2], [3], [4]. Besides, RSUs also improve the perception of autonomous driving vehicles for detecting pedestrians, computing traffic density, recognizing sharp turns, etc. Standards organizations [5], academic communities [6], and governments [7] focus on the development of RSUs to achieve full autonomous driving, namely, light map technology. By integrating vehicles, RSUs, and clouds into a vehicle edge computing (VEC) network, vehicles in the network can utilize environmental information provided by RSUs to strengthen service capacities in autonomous driving [8].