Closed-Loop Reinforcement Learning Based Deep Brain Stimulation Using SpikerNet: A Computational Model

Clinical deep brain stimulation (DBS) has proven effective in treating neurological and neuropsychiatric conditions such as Parkinson's disease and obsessive compulsive disorder. However, current DBS systems operate in open-loop conditions, using stimulation that is agnostic to patient symptomology. Furthermore, current DBS systems require precise parameter tuning for effective stimulation and may require retuning due to device age and long term neural adaptation. The relatively simple control algorithms of current gold-standard closed-loop and adaptive DBS systems sometimes have better clinical outcomes but do not track the patient's dynamic brain states and may be associated with interruptions of voluntary movement. We recently introduced SpikerNet, a closed-loop neuromodulation platform which uses deep reinforcement learning to track brain state changes and actively tune stimulus parameters in real time to achieve targeted neural firing patterns, with the ability to adapt to changing neural dynamics. In this paper, we evaluate SpikerNet in a computational basal ganglia-thalamocortical model of DBS to assess run time and stimulation dynamics. We find that SpikerNet rapidly finds stimulus parameters to restore dopamine depleted activity back to naturalistic states, is stable against rapid and drastic changes in the neural environment, and generates novel stimulation hypotheses for evaluation in preclinical models to derive novel stimulation regimes.