Neuromorphic Spikes Generated by the Retinal Circuit Emulator for Training 3-Dimentinal Convolutional Neural Networks to Reconstruct Natural Images

The vertebrate retina extracts vital visual information in the outside world to encode them into spatiotemporal sequences of the spike signals in a dynamic way. The visual cortical areas are considered to decode those spikes to obtain/construct necessary visual information. Recent computational studies made attempts to re-construct natural images from biological retinal spike sequences by incorporating artificial neural networks as well as signal processing based on top-down design in their decoder models. As the datasets for training such a neural network, artificial retinal spike sequences generated by the software simulation with simple numerical models, such as the linear-nonlinear-Poisson model, have been utilized. However, those frame-based software simulations might be insufficient for simulating spike firing in some subtypes of the retinal output neurons. Therefore, as a complementary approach, we propose to utilize the previously developed hardware emulator of vertebrate retinal circuits. The model parameters of retinal circuitries in the emulator were tuned for a certain subtype of the mouse retinal ganglion cells, and datasets of the neuromorphic spatiotemporal spike sequences in response to stepwise presentations of natural images were generated. The present study demonstrated that these datasets were useful for training three-dimensional convolutional neural networks to reconstruct the natural images.