Computation of deep belief networks using special-purpose hardware architecture

The computation of deep belief networks (DBNs) requires a large number of arithmetic operations, which can be handled only by arithmetic operators. However, the operators are utilized only a small fraction of the time (1-5%) when they are computed by general-purpose computers. In this paper, a special-purpose hardware architecture that computes DBNs using a large number of arithmetic operators with a utilization rate greater than 60% is proposed. On the basis of neuron machine architecture, the computation units in the system are controlled according to stage operation table, which specify the sequence of the computation stages; thus, the complicated procedure of the DBN can be carried out in hardware. Moreover, the usage of the memory space is considerably improved by using offset addressing. The proposed schemes are implemented on a hardware simulator coded in MATLAB and on an FPGA chip. The full source code of the hardware simulator is available at a website. The readers can execute the code on the fly and reproduce the proposed schemes. The FPGA implementation achieves a lower computational time by a factor greater than 100 compared to a PC.

I. Introduction

Recently, deep belief network (DBN) models have captured significant interest from the machine-learning community and brought the resurgence of neural networks. The superior ability of DBNs in learning features that capture higher-order correlations in unsorted data has allowed DBNs to be applied successfully in many application domains such as classification, data mining, recognition, and robotics [1].