Boosting Reinforcement Learning in Competitive Influence Maximization with Transfer Learning

Companies aim to promote their products under competitions and try to gain more profit than other companies. This problem is formulated as a Competitive Influence Maximization (CIM). Recently, a reinforcement learning has been used to solve the CIM problem, that is, to find an optimal strategy against competitor in order to maximize the commutative reward under the competition from other agents. However, reinforcement learning agents require huge training time to find an optimal strategy whenever the settings of the agents or the networks change. To tackle this issue, we propose a transfer learning method in reinforcement learning to reduce the training time and utilize the knowledge gained on source network to target network. Our method relies on two ideas, the first one is the state representation of the source and target networks in order to efficiently utilize the knowledge gained on source network to target network. The second idea is to transfer the final Q-solution of source network while learning on the target network. We validate our transfer learning method in similar or different settings of source and target networks while competing against the competitor's known strategies. Experimental results show that our proposed transfer learning method achieves similar or better performance as a baseline model while significantly reducing training time in all settings.

I. Introduction

Competitive Influence Maximization (CIM) problem considers multiple parties offering same or similar products [1] and compete for buyers or users in the social network. A user will choose only one of the products. The parties, therefore, need to compete to attract as many users as possible in order to gain profit. Most existing studies extended the traditional Influence Maximization (1M) models to formulate the CIM problem [2]–[4]. Recently, a reinforcement learning (RL) based model is proposed to solve the CIM problem while selecting a fixed number of the node(s), that is, a single node, in each around for influence propagation [5]. Nevertheless, existing solutions in the literature require to train the model in per-setting per network basis. That is, whenever the target network changes (different networks, different diffusion models, …) or the settings of agents change (budget, deadline, …), the model must be re-trained in order to reach acceptable