Exploratory Bandit Experiments with “Starter Packs” in a Free-to-Play Mobile Game

This paper explores the application of bandit methods for the assignment of “starter packs” to new players in a free-to-play mobile game environment. Leveraging an online reinforcement learning system, the study aims to strategically assign starter packs to players from different country and device segments. The architecture of the online experimentation system enables real-time decision-making processes and continuous model tuning. The cold start problem is addressed by seeding the bandit with a prior offer policy informed by institutional expertise. Offline evaluation on a dataset derived from a previous AB test conducted by the company and two online experiments assess the abilities of bandit methods to assign starter packs to new players in this environment. While personalization of starter pack assignment in contextual data (country and device segments) is not achieved, a bandit with a conversion-reward changes the prior institutional policy and lowers the average effective sales price of starter packs to new players. The bandit’s policy achieves an indicative lift in per-user revenue, repeat purchasing, and player retention compared to a holdout group with a naive policy. This research contributes to the field of monetization strategies in mobile free-to-play games, emphasizing the design of systems for online personalization and revenue optimization.