Mini-Batch Training along Convolution Windows for Representation Learning Based on Spike-Time-Dependent-Plasticity Rule

This paper presents a mini-batch training methodology along convolutional windows for layer-wised STDP unsupervised training on convolutional layers in order to shorten the training time of spiking neural networks (SNNs). SNN is a third-generation neural network that uses an accurate neuron model compared to rate-coded models used in conventional artificial neural networks (ANNs). The mini-batches of input convolution windows are convoluted at once. Then, the input, output, and current filter generate a batch of weight updates at once. This system reduces overheads of library calls or GPU execution. The batch processing methodology leads more significant and extensive models to be trained in ANNs, while many evaluations of direct SNN training methodologies are limited to smaller models. Currently, training large-scale models is virtually impossible. We evaluated the mini-batch processing effect on training speed and feature extraction power against various mini-batch sizes. The result showed that a larger mini-batch size enables us to utilize GPUs effectively, maintaining comparable feature extraction power. This research concludes that mini-batch training along convolution windows reduces training time by STDP training rule.