Input Excitation Optimization for Estimating Battery Electrochemical Parameters using Reinforcement Learning

Parameter estimation is an important topic in battery management, and the quality of an input excitation has critical impact on estimation accuracy. The design of optimal excitation has previously been formulated as determining a time sequence of inputs that maximizes a certain criterion related to estimation accuracy. The resultant problem suffers from susceptibility to modeling errors and other uncertainties, as well as high computational complexity due to the length of the input sequence to be optimized. In this work, we formulate input excitation design as an optimal control problem with the goal of obtaining a closed-loop control policy for current generation that is robust to uncertainties. To facilitate the solution of the problem, we envision input generation as a Markov Decision Process with dynamics described by certain states, and leverage reinforcement learning (RL) to obtain the optimal policy, which maximizes the Fisher information of the target parameter to be estimated. We demonstrate the result for a key battery electrochemical parameter and compare with the result generated through direct input sequence optimization. It is shown that the RL approach can produce competitive results under an accurate system model, while significantly outperforming the conventional approach under the presence of model (parameter) uncertainty. The proposed RL-based approach also has the potential of enabling input excitation optimization for models with high complexity and nonlinearity, which is extremely difficult, if not infeasible, for the conventional method.