HPCNeuroNet: Advancing Neuromorphic Audio Signal Processing with Transformer-Enhanced Spiking Neural Networks

Our work addresses the challenge of efficiently processing complex audio signals in noisy environments using neuromorphic computing. We introduce HPCNeuroNet, an innovative architecture that combines Spiking Neural Network (SNN) and Transformer. Our architecture, HPCNeuroNet, employs the Short-Time Fourier Transform (STFT) for time-frequency representation, Transformer embeddings for dense vector generation, and SNN encoding/decoding mechanisms for spike train conversions. Our solution utilizes NVIDIA's RTX 3060 GPU and Intel's Core i9 12900H CPU, with a novel implementation on the Xilinx VU37P FPGA platform for enhanced energy efficiency and real-time processing capabilities. The proposed accelerator achieves a throughput of 71.11 Giga-Operations Per Second (GOP/s) throughput with a 3.55 Won-chip power consumption. Comparing the proposed accelerator to off-the-shelf devices and stateof-the-art implementations reveals its apparent energy efficiency and flexibility advantages.