Modular fuzzy-reinforcement learning approach with internal model capabilities for multiagent systems

To date, many researchers have proposed various methods to improve the learning ability in multiagent systems. However, most of these studies are not appropriate to more complex multiagent learning problems because the state space of each learning agent grows exponentially in terms of the number of partners present in the environment. Modeling other learning agents present in the domain as part of the state of the environment is not a realistic approach. In this paper, we combine the advantages of the modular approach, fuzzy logic and the internal model in a single novel multiagent system architecture. The architecture is based on a fuzzy modular approach whose rule base is partitioned into several different modules. Each module deals with a particular agent in the environment and maps the input fuzzy sets to the action Q-values; these represent the state space of each learning module and the action space, respectively. Each module also uses an internal model table to estimate actions of the other agents. Finally, we investigate the integration of a parallel update method with the proposed architecture. Experimental results obtained on two different environments of a well-known pursuit domain show the effectiveness and robustness of the proposed multiagent architecture and learning approach.

I. Introduction

Multiagent systems related research is an emerging subfield of distributed artificial intelligence, which aims at providing both: principles for the construction of complex systems involving multiple agents and mechanisms for the coordination of independent agents' behavior. The most important reason to use multiagent systems is to have a more natural modeling for real-life domains that require the cooperation of different parties. In particular, if there are different people with different perspectives or organizations with different goals and proprietary information, then a multiagent system is needed to handle their interaction [1]. Multiple agents are recognized as crucial for many real-world problems, such as engineering design, intelligent search, medical diagnosis, robotics, etc. [15].