Reinforcement Learning for Optimizing Age of Information and Energy Consumption in Misson-critical IoT

The emerging Internet of Things (IoT) play a vital role in remote monitoring. Age of Information (AoI) is an effective metric defined as the elapsed time since the generation of the packet by the transmitter. In this paper, we study a remote monitoring system consisting of a scheduling node and some sensor nodes. The scheduling algorithm strives to find the optimal trade-off between minimizing the sum of the expected AoI, the AoI violation probability, the transmission energy consumption of the sensor nodes and the cost of sensor node replacement due to battery drainage. We model the problem as a Markov Decision Process (MDP) with finite state and action spaces, and utilizes the certainty equivalence to approximate the stochastic reward. An actor-critic-based algorithm is proposed. Simulation results show that compared with the random scheduling policy and the policy which only considers minimizing sum of AoI and AoI violation probability, the proposed algorithm can optimize both AoI and energy more effectively.