Opponent Modeling Based Dynamic Auction for UAV-Assisted Task Offloading

This paper proposes a dynamic auction based resource trading scheme for unmanned aerial vehicle (UAV) assisted mobile edge computing (MEC) network, where multiple mobile users (MUs) compete to purchase the task offloading service from the UAV by bidding strategically. The UAV-assisted MEC server aims to maximize its long-term profit at the least energy costs by adaptively adjusting its service location via path planning. With dynamic task arrivals, each MU optimizes its sequential bidding strategy to minimize its task computation delay and payment under the budget constraint. Since each MU's remaining budget is a private information, we model this dynamic auction as a partially observable stochastic game, where the MU with the highest bid and sufficient remaining budget in each round wins the UAV's exclusive computation service and pays the second-highest price to the UAV. To tackle the issue of non-stationarity in a multi-agent environment, we propose an opponent modeling based reinforcement learning algorithm, namely neural fictitious self-play with dueling double deep Q Network (NFSP-D3QN), to optimize the trading policies for both the UAV and MUs. Specifically, the proposed algorithm incorporates fictitious self-play to implicitly predict the opponents' behavior, assisting each agent's sequential decision making in a fully decentralized manner. Simulation results demonstrate that, compared with the baseline algorithm, our proposed algorithm enhances both the UAV's and MUs' utilities, achieving efficient utilization of computing resources.