Enhancing Fog Computing through Intelligent Reflecting Surface Assistance: A Lyapunov Driven Reinforcement Learning Approach

The explosive growth of mobile devices in the Internet of Things (IoT) has significantly increased the demand for fog computing (FC). As a key component in optimizing wireless communication environments, intelligent reflecting surfaces (IRS) have garnered considerable attention. This paper develops an online optimization model for IRS-assisted FC, implemented across multiple cells with computational nodes. We propose a Lyapunov-function-based, space aggregation-aided proximal policy optimization (LSAPPO) algorithm to address the challenges of online optimization in IRS-assisted FC environments. Our approach introduces a reinforcement learning algorithm that employs a proximal policy optimization (PPO) agent, enhanced by Lyapunov drift-plus-penalty Optimization. The space aggregation method efficiently consolidates excessive channel state information (CSI) and decision variables into a manageable set of parameters, thereby simplifying the computational framework. Numerical results demonstrate that our algorithm outperforms established benchmarks, highlighting its effectiveness in complex wireless environments.

I. Introduction

In the era of the Internet of Things (IoT), there has been an explosive growth in mobile devices such as wearable technology, smartphones, and unmanned mobile units, all aimed at providing flexible IoT services [1]. However, these IoT mobile devices are often constrained by their size and cost, which limit their computational resources [2]. This limitation poses a significant challenge, particularly with the emergence of computationally intensive applications such as pattern recognition and cognitive assistance. Typically, IoT devices rely on computational offloading to handle these demanding tasks. Fog computing (FC) has emerged as a popular paradigm that combines the high computational power of cloud computing with the low latency benefits of edge computing, making it an ideal solution for computational offloading [3].