Intelligent Knowledge Distribution: Constrained-Action POMDPs for Resource-Aware Multiagent Communication

This article addresses a fundamental question of multiagent knowledge distribution: what information should be sent to whom and when with the limited resources available to each agent? Communication requirements for multiagent systems can be rather high when an accurate picture of the environment and the state of other agents must be maintained. To reduce the impact of multiagent coordination on networked systems, for example, power and bandwidth, this article introduces two concepts for the partially observable Markov decision processes (POMDPs): 1)
action-based constraints
that yield constrained-action POMDPs (CA-POMDPs) and 2)
soft probabilistic constraint satisfaction
for the resulting infinite-horizon controllers. To enable constraint analysis over an infinite horizon, an unconstrained policy is first represented as a finite-state controller (FSC) and optimized with policy iteration. The FSC representation then allows for a combination of the Markov chain Monte Carlo and discrete optimization to improve the probabilistic constraint satisfaction of the controller while minimizing the impact on the value function. Within the CA-POMDP framework, we then propose intelligent knowledge distribution (IKD) which yields per-agent policies for distributing knowledge between agents subject to interaction constraints. Finally, the CA-POMDP and IKD concepts are validated using an asset tracking problem where multiple unmanned aerial vehicles (UAVs) with heterogeneous sensors collaborate to localize a ground asset to assist in avoiding unseen obstacles in a disaster area. The IKD model was able to maintain asset tracking through multiagent communications while only violating soft power and bandwidth constraints 3% of the time, while greedy and naive approaches violated constraints more than 60% of the time.