Deadline-Aware Deep-Recurrent-Q-Network Governor for Smart Energy Saving

Complex cyber-physical-social systems (CPSS) consist of battery-supplied devices with low energy consumption requirements. It is essential to maintain the timing performance of computing or communication tasks while saving the device energy. Linux OS provides built-in frequency governors for power management. However, these governors are not able to incorporate the timing requirements of the application for decision-making and cannot adapt the power management decision to the specific application on target devices. This paper presents an intelligent Linux frequency Deep-Recurrent-Q-Network (DRQN) governor for dedicated applications with deadline requirements running on CPSS devices through machine learning. Although machine learning algorithms have made considerable breakthroughs in recent years, deploying them to real small devices is challenging because of the computational overhead. To tackle the computation overhead problem, an interactive learning framework is designed where the DRQN model performs only the lightest inference in the kernel while using the online data (time-series) to learn and update itself in real-time at the user level. The governor is tested on both standalone devices and networked devices. The experiment shows that DRQN can self-develop tradeoff policy to meet the user's need with low overhead. The energy saved by DRQN ranges from 7% to 33% for various deadlines.