Evaluating Markov Decision Process as a Model for Decision Making Under Uncertainty Environment

Soccer simulation is a suitable environment for implementing and testing modern multi-agent artificial intelligence distributed algorithms. Decision making under uncertainty conditions is one of the main problems in multi-agent systems. Uncertain and stochastic factors affect the performance of the action and make the agent unable to take the appropriate task. A high complexity of such environment causes decision making on the basis of traditional method (hard code) to be absolutely difficult, so new approaches of decision making such as Markov decision process (MDP) and neural networks are used. In our research, the soccer 3D simulation is used as a test bed for multi-agent systems. In this paper, the Markov decision process is used as a strategy for decision making of soccer 3D agents, and also a novel policy is proposed in our MDP method. The simulation results are used for performance evaluation of our proposed approach in comparison with traditional method in situations that agents' sense/act abilities have a lot of noise involved, the simulation results show that in our MDP method, the agents make better decision and carry out better action which help to reach the goal of plan.

1. Introduction

In the resent decades, researchers of Multi-Agent system consider decision making as an interesting problem. In this problem goal is defined as a state with highest utility and agent must choose action or a set of actions to reach goal.