A Hybrid Heterogeneous Neural Network Accelerator based on Systolic Array

Spiking neural networks (SNNs) and artificial neural networks (ANNs) are two methods to achieve artificial intelligence. Realizing the long-term and ambitious ideal of a universal artificial intelligence platform requires hardware that supports both SNN and ANN models. ANNs, such as convolutional neural networks (CNNs), are adept at extracting features and have achieved outstanding achievements in many fields. Due to the event-driven processing paradigm, SNNs can achieve competitive accuracy with minimal power usage. The weight accumulation of SNN and ANN is similar, but the activation function is significantly different. How to integrate SNN and ANN on one hardware platform is a challenge. In this paper, we propose a hybrid heterogeneous neural network accelerator that can support both SNN and ANN models. It also has good compatibility with the binary neural network (BNN) models. Our design is implemented on Xilinx XCKU115 FPGA, which can achieve the peak performance of 48.51GSOP/s (SNN) and 26.38GOP/s (ANN). The energy efficiency is 25.8GSOP/W (SNN) and 12.05GOP/W (ANN). Moreover, it supports the SNN-ANN fusion mode, where each row of processing elements (PEs) can independently perform SNN or ANN computation, allowing the design to fully leverage the respective strengths of SNN and ANN.