A Deep Reinforcement Learning Scheduler with Back-filling for High Performance Computing

It is becoming widely accepted that model-based job scheduling strategies are becoming inadequate due to an ever increasing complexity of system infrastructure, highly variable workloads and user requirements. Research into reinforcement learning-based approaches for job scheduling is a compelling and rapidly advancing direction. In this work we propose a novel deep reinforcement learning approach to the classic job scheduling problem. Our approach uses a bounded observation space, allows the learning agent to postpone scheduling to later scheduling cycles and we show experiments that give insights into its performance with respect maximizing utilization and minimizing wait times. Our results show competitive outcomes that indicate our approach is a favourable framework for job scheduling in more complex systems.