Adaptive Directional Neighbor Discovery Schemes in Wireless Networks

The use of fully directional links in ad-hoc networks improves the overall network performance but complicates the process of neighbor discovery. In most of the state-of-the-art schemes, nodes switch their direction either randomly or in a predefined sequence, without considering the result of past discovery attempts in each sector. This introduces high discovery latency and severe long tail problem, especially for high density networks and/or narrow beams. To overcome these limitations, we propose two novel decentralized, and low complexity reinforcement learning-based neighbor discovery schemes in this paper. In these schemes, the neighbor discovery is mapped to a stochastic multi-player game, where each node independently adjusts its policy via the Q-learning based scheme to minimize the discovery latency. Effectiveness of the proposed schemes is assessed through numerical results, and it is observed that these proposed schemes are able to achieve a significantly faster network-wide neighbor discovery, while incurring low overhead and computational complexity.