Efficient Parameter Tuning for Multi-agent Simulation Using Deep Reinforcement Learning

This study proposes a reinforcement-learning-based method for efficient parameter tuning in multi-agent simulations (MAS). Usually, MAS has a high computational burden because of the agents involved; thus, it is important to tune its parameters efficiently. Our proposed method is centered around actorcritic-based reinforcement learning methods, such as deep deterministic policy gradient (DDPG) and soft actor-critic (SAC). In addition, this study proposes three additional components: an action converter (only for DDPG), a redundant full neural network actor, and a seed fixer. As an experiment, a parameter-tuning task is employed in an artificial financial market simulation. A Bayesian estimation-based method is used as the baseline. The results show that our model outperforms the baseline in terms of tuning performance, indicating that additional components of the proposed method are essential. Moreover, the critics of our models effectively function as surrogate models, that is, as an approximate function of the simulation, which allows the actor to appropriately tune the parameters.