MetaplasticNet: Architecture with Probabilistic Metaplastic Synapses for Continual Learning

Metaplasticity, the activity-dependent modification of synaptic plasticity, is an important technique for mitigating catastrophic forgetting in neural networks. Often, continual learning models with metaplasticity require compute-intensive training. In this research, we propose a probabilistic metaplastic synapse with discrete hidden states that alleviates the computational cost. We implement a digital architecture of the network with on-chip training to achieve further power savings. Results show upto ~ 22% and ~ 21% improvement in mean accuracy for Split-MNIST and sequential MNIST-FMNIST benchmarks respectively, compared to previous metaplasticity models. Simulations of the full digital architecture show ~ 53× lower power consumption per weight update with similar accuracy as gradient-based network counterparts.