Control of an Under-Actuated Cartpole with an Evolving Neural Topology Using Synaptic Bellman Equation

This paper proposes a synaptic Q-learning algorithm, for the classical cartpole control problem where the Bell-man equations are incorporated at the synaptic level. This enables the iterative evolution of the network topology as a directed graph during training. Topology evolution, in conjunction with mixed-signal computation, leverages the optimization of the number of neurons and synapses that could be used to design spike-based reinforcement learning accelerators. The proposed architecture has the potential to reduce resource utilization on board, hence creating more compact application-specific neuromorphic ICs.