Thompson Sampling-Based Dynamic Spectrum Access in Non-Stationary Environments

In dynamic spectrum access (DSA), unlicensed secondary users (SUs) estimate the idle probabilities of primary channels by using historical sensing results and opportunistically access the channel with the highest idle probability for transmission. Due to the rapid traffic changes and irregular user mobility, primary channels can be highly dynamic and their idle probabilities are generally time-varying. In this paper, we investigate DSA in non-stationary environments. Specifically, we consider two channel state models, i.e., the non-stationary Bernoulli model with a time-varying mean and the non-stationary Markovian model with a time-varying transition matrix. For the single-SU scenario, we propose a Thompson sampling based method with a change detection technique, which is capable of detecting the variations of channel statistics and adjusting the channel access strategy accordingly. For the multi-SU scenario, we propose a Thompson sampling based collision alleviation method to coordinate the transmissions of SUs, which does not need any prior protocol or information exchange among SUs. Numerical results show that the proposed methods outperform the existing ones in terms of successful transmission ratio in various network settings.