Gain scheduling control with Markov transition models

Adaptive control based on multiple Markov transition models (MTM) is considered. MTM's are constructed off-line for a set of candidate plant models, and an optimal controller is designed for each plant model. Frequentist information obtained from measured data is compared on-line with the state transitions predicted by the plant models. The best model is selected based on Kullback-Leibler distances averaged over a time window. The manipulated input of the plant is obtained from the control policy associated with the selected model. Two simulations are conducted to illustrate the approach in adaptive (gain scheduling) control of a non-linear nonminimum phase van der Vusse continuous stirred tank reactor (CSTR). The first simulation concerns an ideal CSTR when the tempeature changes, the second simulation considers control of a non-ideal CSTR where streams in different temperatures are sampled randomly.