F3D: Accelerating 3D Convolutional Neural Networks in Frequency Space Using ReRAM

3D convolutional neural networks (CNNs) are widely deployed in video analysis. Fast algorithms such as fast Fourier transforms (FFTs) are gaining popularity in reducing computation complexity for their superior capability of replacing convolutions with simpler element-wise multiplications. Conventional frequency-domain dedicated accelerators employ memory hierarchy organization for high throughput but at the expensive costs of a significant amount of data movements and energy consumptions. This paper presents F3D, a processingin-memory frequency-domain accelerator using resistive random access memory (ReRAM). F3D supports frequency-domain complex number multiplications directly in ReRAM-based crossbar architecture. We alleviate the overheads of redundant data movements in ReRAM-based complex number multiplications by data reuse and the inherent symmetry of inputs in the frequency space. Evaluation results demonstrate that F3D outperforms state-of-the-art accelerators with significant improvements in performance and energy efficiency.