Optimal Preemption Policy for Age of Information Minimization with Known Packet Length

With the requirement of timeliness increasing, data processing policy should be carefully designed to tackle arrivals. This paper mainly studies the Age of Information (AoI) in data processing system, where packets are generated by a source and processed by a server with known packets' length upon arrival. We aim to minimize the average AoI by deciding either to preempt the current packet or not when a new packet arrives. For the given distributions of inter-arrival time and packets' length, the problem is formulated by Markov Decision Process (MDP) and solved via value iteration. Without prior knowledge of the distributions, we apply Reinforcement Learning (RL) algorithms to learn the policy online. Through simulation experiments, it is revealed that the obtained optimal strategy by MDP greatly reduces the average AoI compared with baseline policies. Further, the RL algorithms have a good performance in solving this problem. The average AoI of RL policies are just slightly higher than those of MDP.