Resilient Cloud Control System: Dynamic Frequency Adaptation via Q-learning

Traditional control systems face challenges in managing high data loads and computing power, prompting the evolution of Cloud Control Systems (CCS)-a fusion of Networked Control Systems (NCS) and cloud computing. Despite offering manifold advantages, CCS encounters hurdles in navigating the dynamic cloud environment characterized by fluctuating workloads, rendering static frequency settings inefficient. Moreover, the optimal utilization of cloud resources poses a pivotal challenge within CCS operations. To address these, the paper proposes a resilient CCS by adapting system frequency dynamically. Leveraging Q-learning, the approach measures Round Trip Time (RTT) and system output errors, dynamically adjusting the system's frequency to minimize control costs, optimize performance within the dynamic cloud environment, and achieve resource frugality, minimizing resource usage. Through real testbed experiments, this paper evaluates and analyzes the effectiveness of the proposed method, aiming to establish an adaptive and efficient control framework aligned with evolving cloud dynamics.