Parameter Tuning Method for Multi-agent Simulation using Reinforcement Learning

This study proposes a reinforcement-learning-based method for efficient parameter tuning in multi-agent simulations (MAS). MAS usually has a high computational burden because of the agents; thus, it is important to tune its parameters efficiently. Our proposed method is based on a deep deterministic policy gradient (DDPG) and has three additional components proposed in this study: an action converter, a redundant full neural network actor, and a seed fixer. As an experiment, we employed a parameter-tuning task in an artificial financial market simulation. A Bayesian estimation-based method was employed as a baseline. The results show that our model tends to outperform the baseline in terms of tuning performance and indicate that the three additional components of the proposed method are essential. Moreover, the critic of DDPG effectively functions as a surrogate model, that is, as an approximate function of the simulation, which allows the actor to appropriately tune the parameters.