Blind Beamforming for Intelligent Reflecting Surface: A Reinforcement Learning Approach

The beamforming problem of intelligent reflecting surface (IRS) has been extensively considered from an optimization perspective assuming that channel state information (CSI) is available. However, the reality is that the existing prototypes seldom follow this model-based approach because channel estimation is technically difficult and costly for the network protocols and hardware to date. A recent trend is to perform beamforming blindly without channel knowledge, e.g., the so-called CSM method [1], [2]. This work looks at blind beamforming from a reinforcement learning point of view. We first show that CSM boils down to a special case of the greedy algorithm in the reinforcement learning context. We analyze the resulting cumulative regret, and further propose an upper approximation to facilitate the optimization of the exploration probability. Moreover, we show that a gradient sampling scheme can improve the efficiency of reinforcement learning as compared to the uniform sampling scheme used in CSM. Finally, we validate the performance advantage of the proposed methods in a prototype system.