E-STDP: A Spatio-Temporally Local Unsupervised Learning Rule for Sparse Coded Spiking Convolutional Autoencoders

Sparse coding algorithms can be used to train a network to perform compression and image reconstruction similar to autoencoders. These algorithms have allowed unsupervised learning of biologically plausible image filters in Spiking Neural Networks (SNNs) that can be used for image classification tasks. These methods have the advantage of being more power efficient due to their sparsity and spiking nature. However, in the absence of a temporally local learning rule, the training step has to be done offline in a separate conventional computing framework. We propose a spike-based temporally local update rule that implements sparse coding in SNNs. We trained a Convolutional Neural Network (CNN) which achieves a test accuracy of 98.35% on the MNIST dataset. Furthermore, it improves the power efficiency by representing an MNIST digit by around 10,000 spikes which is 4x more efficient representation, in terms of bits required, compared to a standard non-spiking CNN. Since the updates are both temporally and spatially local, it also performs the required learning within a small multiple of 10,000 operations per sample compared to around 1,870,000 operations per sample required to train a typical non-spiking CNN having a similar structure. Such a spatio-temporally local learning rule opens the way for implementation on neuromorphic hardware without the systemic and operational inefficiencies of global information broadcast.