QL-TSCH-plus: A Q-learning Distributed Scheduling Algorithm for TSCH Networks

Addressing the requirements of Industrial Internet of Things (IIoT) in Industry 4.0, the Time Slotted Channel Hopping (TSCH) protocol of the IEEE 802.15.4e amendment has been proposed. However, the lack of a defined scheduling procedure in the standard remains an open research area. Existing reinforcement learning-based scheduling proposals demonstrate great potential for this technique due to the ongoing observations within the network environment. Beneficial for real-world scenarios where network conditions are volatile and unpredictable. This work presents QL- TSCH-plus, an enhancement of the existing QL- TSCH scheduler that reduces energy consumption by adapting the Action Peeking mechanism to a distributed scheme. Instead of continuously listening to neighboring nodes communication, QL- TSCH-plus allows nodes to broadcast the learned transmission slots for updating the Action Peeking Tables and allocating reception slots, reducing energy use by up to 47% compared to QL-TSCH. This novel approach also maintains reliability and timeliness, demonstrating significant potential for efficient scheduling in TSCH networks, making it suitable for the IIoT.