Learning-based Scheduling for Information Accuracy and Freshness in Wireless Networks

We consider a system of multiple sources, a single communication channel, and a single monitoring station. Each source measures a time-varying quantity with varying levels of accuracy and one of them sends its update to the monitoring station via the channel. The probability of success of each attempted communication is a function of the source scheduled for transmitting its update. Both the probability of correct measurement and the probability of successful transmission of all the sources are unknown to the scheduler. The metric of interest is the reward received by the system which depends on the accuracy of the last update received by the destination and the Age-of-Information (AoI) of the system. We model our scheduling problem as a variant of the multi-arm bandit problem with sources as different arms. We propose 4 bandit policies, namely, Q-ETC, Q-∊-greedy, Q-UCB, and Q-TS suitably adjusted to our system model via simulations. In addition, we provide analytical guarantees of 2 of these policies, Q-ETC, and Q-∊-greedy. Finally, we characterize the lower bound on the cumulative regret achievable by any policy.