Deep Phasor Networks: Connecting Conventional and Spiking Neural Networks

Artificial neural networks (ANNs) are powerful but require many orders of magnitude more energy than biological systems capable of solving similar tasks. One critical difference is that ANN units communicate using continuous signals, as opposed to the binary spike events employed by biological networks. In this work, we extend standard ANN design by building upon an assumption that neuronal activations correspond to the angle of a complex number lying on the unit circle, or ‘phasor’- the same domain employed by the Fourier Holographic Reduced Representation (FHRR) Vector-Symbolic Architecture (VSA). Each layer in such a network produces new activations by taking a weighted superposition of the previous layer's phases and calculating this sum's phase value. This generalized architecture allows models to reach high accuracy and carries the singular advantage that mathematically equivalent versions of the network can be executed with or without regard to a temporal variable. Importantly, the value of a phase angle in the temporal domain can be sparsely represented by a periodically repeating series of delta functions or ‘spikes.’ We demonstrate the atemporal training of a phasor network on standard deep learning tasks and show that these networks can then be executed in either the traditional atemporal domain or spiking temporal domain with no conversion step needed. This provides a novel basis for constructing deep networks which operate via temporal, spike-based calculations suitable for low-energy neuromorphic computing hardware.