Multi-DMC: Deep Monte-Carlo with Multi-Stage Learning in the Card Game UNO

Games can serve as important benchmarks for evaluating artificial intelligence (AI), and many game AIs have been invented, such as AlphaGo, Libratus, OpenAI Five, Suphx, and DouZero. For UNO, a popular shedding-type card game, the AI faces imperfect-information challenges, large state space, and sparse rewards. The existing methods only demonstrate the promising feasibility of classic reinforcement learning in the UNO game, which is still far from being able to compete with human players. Based on the Monte-Carlo method and deep neural networks, we combine it with Multi-Stage Learning during the training process, called Multi-DMC. We conduct experiments on the Two-vs-Two cooperative competition version which is a commonly used version in tournaments. With 4 billion training steps, Multi-DMC in UNO outperforms all the existing baselines, and it achieves more than a $99 \%$ winning rate against Random agents and $85 \%$ against Rule-Based agents. Meanwhile, we find that Multi-DMC is more competitive than pure self-game DMC-Self. The project page is at https://github.com/LinsLab/UNO.