Training-to-Learn with Memristive Devices

Memristive crossbar arrays are promising non-von Neumann computing technologies to enable real-world, online learning in neural networks. However, their deployment to real-world learning problems is hindered by their non-linearities in conductance updates, variation during operation, fabrication mismatch and the realities of gradient descent training. In this work, we show that, with a phenomenological model of the device and bi-level optimization, it is possible to pre-train the neural network to be largely insensitive to such non-idealities on learning tasks. We demonstrate this effect using Model Agnostic Meta Learning (MAML) and a differentiable model of the conductance update on the Omniglot few-shot learning task. Since pre-training is a necessary procedure for any on-line learning scenario at the edge, our results may pave the way towards real-world applications of memristive devices without significant adaption overhead.