Extremum Seeking for Creating Optimal Feedback Controls of Unknown Systems by Tuning Basis Functions

We consider the problem of optimal feedback control of unknown dynamic systems. Optimal feedback control is important for systems with time-varying initial conditions, which cannot rely on feedforward controllers. For example, components in particle accelerators are turned on and off hundreds of times per second, with pulse widths of ~1 ms and repetition rates >100 Hz, whose dynamics and initial conditions vary slowly over time due to external disturbances, such as temperature fluctuations. Our approach allows us to quickly learn an optimal feedback control, which can then be applied for all initial conditions even as the system begins to drift and change. For linear systems we reproduce the cost minimizing linear quadratic regulator (LQR) optimal controller that could have been designed had the system been known.