A Dynamic Surrogate Gradient Function for Memristive Spiking Neural Networks

Spiking Neural Networks (SNNs) have some dis-tinct advantages in addressing temporal data by simulating the information transmission mode of brain neurons. Due to the nondifferentiable behavior of spike emission, previous works try to replace the heaviside function with some static approximation functions to solve this problem. However, such static surrogate gradient approach can potentially introduce errors in gradient propagation. Therefore, in this paper, we first propose a dy-namic surrogate gradient function based on Tanh to reduce errors during gradient propagation. Furthermore, we propose a new synapse simulation method by using memristor crossbar arrays in SNNs to solve the memory wall problem. Finally, we conduct sufficient experiments on multiple datasets to verify the effectiveness of the dynamic surrogate gradient function in SNN. Experiment results show that our proposed algorithm can achieve high performance compared with the previous works.