Control of Non-Equilibrium Dynamics in Communication Networks

We propose a novel, dynamical systems-based approach for analysis and control of the complex dynamic behavior arising in communication networks. Specifically, we consider the Active Queue Management (AQM) problem and study the non-equilibrium behavior observed as a result of the interaction between deterministic queueing and nonlinear flow-control dynamics using a stochastic characterization. The asymptotic dynamics are interpreted using invariant measures of certain stochastic operators. For computational tractability, we use set-oriented numerical methods to construct finite-dimensional Markov models including control Markov chains and hidden Markov models. Based on the stochastic model constructed, we pose and solve the AQM control problem using Markov Decision Processes (MDPs). The framework developed is demonstrated through a numerical study of an example AQM scheme, which shows persistent non-equilibrium queue behavior under the optimal control strategy.