A Comparative Study of Q-Learning and DQN on Large Grid World Environments

Reinforcement learning algorithms have revolutionized autonomous decision-making in various domains. In this paper, we compare Q-learning and DQN for solving a 100x100 grid model of a real-world problem. Our investigation leverages the technique of coarse coding, enabling efficient state space approximation for the traversal of large environments. This study follows standard evaluation strategies, convergence, epsilon decay dynamics, and algorithmic behaviors in both Q-learning approaches. Our research shows Q-learning with coarse coding converges, whereas DQN has contextual limitations, challenging the one-size-fits-all notion. Finally, we highlight coarse coding’s limitations in Q-learning, underscoring the necessity of fine-grained state representations for optimal paths. Our findings highlight the vital role of algorithm choice in complex, real-world scenarios, adding to the ongoing adaptability discussion of Q-learning techniques.