Traffic Shaping with Deep Q-Networks for Optimizing the Age of Information

The Age of Information (AoI) has emerged as a pivotal metric in quantifying the freshness and relevance of information at the receiver's end. While theoretical advancements have been made in basic queueing systems, practical implementations and dynamic management of AoI remain underexplored. This research aims to bridge this gap by investigating the intricacies of AoI, exploring different forms of the metric, and proposing novel techniques to improve its performance, particularly in terms of the average peak AoI. Complementing traditional metrics like response time and throughput, AoI captures the timeliness of information, influenced not only by generation rates and link characteristics but also by network congestion from various sources. A key aspect that requires further examination is the implementation of dynamic traffic shaping at routers, specifically targeting peak AoI to establish an upper limit on information freshness. However, this problem presents a combinatorial challenge, requiring sequential decision-making for each packet arrival to achieve suitable tradeoffs between dropping packets and allowing them to pass. To address this problem, we explore a machine learning approach utilizing Deep Q-Networks (DQNs) to optimize the decision-making processes. By training the DQN to determine the packet-drop decisions, we enhance AoI management and enable the efficient delivery of timely information.