A Hybrid CMOS-Memristive Approach to Designing Deep Generative Models

Deep learning and its applications have gained tremendous interest recently in both academia and industry. Restricted Boltzmann machines (RBMs) offer a key methodology to implement deep learning paradigms. This brief presents a novel approach for realizing hybrid CMOS-memristive-based deep generative models (DGMs). In our proposed DGM architecture, HfO
_x
-based (filamentary-type switching) memristive devices are extensively used for realizing both computational as well as storage functions, such as: 1) synapses (weights); 2) internal neuron-state storage; 3) stochastic neuron activation; and 4) programmable signal normalization. To validate the proposed scheme, we have simulated two different architectures: 1) deep belief network (DBN) for classification and 2) stacked denoising autoencoder for the reconstruction of handwritten digits from the MNIST data set. The maximum test accuracy achieved by pretraining of the proposed DBN was 92.6%, whereas the best case mean squared error (mse) achieved by pretraining of the proposed SDA network was 0.046. When the proposed model-based weights are used for weight initialization, they offer a significant advantage in terms of learning performance in comparison with randomized initialization.