Coordination in Markov Games with Asymmetric Information

We study coordination in Markov games with asymmetric information. We consider a model where the state consists of different components, each representing the private type of each player. Players' actions depend on their private types and the public observation of past actions. The state components evolve as independent Markov processes conditioned on actions. We propose a solution concept called perfect correlated equilibrium (PCE), realized by a correlation device that observes only the public information of past actions. At time t, the device generates a prescription profile from a commonly known joint distribution, and sends each player a prescription privately before they act. Players are expected to take actions according to the prescriptions at equilibrium by evaluating the suggested prescription at the private types. We introduce “structured” PCE (sPCE), in which the correlation device generates prescriptions based on the common action history through a common belief on the state. We motivate sPCE by showing that any payoff profile induced by a general device can be induced by a structured one. We show that when the correlation device is using structured strategies, players' rationality constraints can be characterized through appropriate Markov decision processes (MDPs). Based on this characterization, we develop a backward dynamic approach, with which one can verify if a structured device is feasible, or even design a structured PCE in a backward recursive manner. Finally, we consider a specific example demonstrating how coordination can improve social welfare.