Predicting Performance using Approximate State Space Model for Liquid State Machines

Liquid State Machine (LSM) is a brain-inspired architecture used for solving problems like speech recognition and time series prediction. LSM comprises of a randomly connected recurrent network of spiking neurons. This network propagates the non-linear neuronal and synaptic dynamics. Maass et al. have argued that the non-linear dynamics of LSM is essential for its performance as a universal computer. Lyapunov exponent (μ), used to characterize the non-linearity of the network, correlates well with LSM performance. We propose a complementary approach of approximating the LSM dynamics with a linear state space representation. The spike rates from this model are well correlated to the spike rates from LSM. Such equivalence allows the extraction of a memory metric (τ
_M
) from the state transition matrix. τ
_M
displays high correlation with performance. Further, high τ
_M
systems require fewer epochs to achieve a given accuracy. Being computationally cheap (1800× time efficient compared to LSM), the τ
_M
metric enables exploration of the vast parameter design space. We observe that the performance correlation of the τ
_M
surpasses that of Lyapunov exponent (μ), (2 - 4× improvement) in the high-performance regime over multiple datasets. In fact, while μ increases monotonically with network activity, the performance reaches a maxima at a specific activity described in literature as the edge of chaos. On the other hand, τ
_M
remains correlated with LSM performance. Hence, τ
_M
captures the useful memory of network activity that enables LSM performance. It also enables rapid design space exploration and fine-tuning of LSM parameters for high performance.