Nanoscale Design of Multi-Layer Perceptrons using Floating-Point Arithmetic Units

Deep learning requires powerful designs to process large volume of data during inference while meeting other metrics of high performance such as efficient hardware, operating frequency, power dissipation and latency. In this paper, the shortcomings of current technologies in implementing the forward propagation of Multi-Layer Perceptrons (MLPs) are first investigated. Subsequently, a novel design at nanoscale that combines parallelization and serialization techniques (so hybrid), is proposed to implement a high performance MLP. This design utilizes single precision floating-point arithmetic units and a fully pipelined scheme; it is very flexible because the hardware is independent of the network topology, except for the number of cycles and therefore, the latency. Simulation results for the MNIST dataset show that compared to existing MLP designs found in the technical literature, the proposed design achieves the least power consumption, the highest operating frequency and accuracy; however, it suffers from a slight increase in latency.