Exploring Memory Access Techniques for Efficient FPGA based 3D CNN Accelerator Design

Recent advancements in 3D Convolutional Neural Network (CNN) architectures have demonstrated superior performance across diverse computer vision tasks, albeit with a trade-off of intense computational and memory demands. Thus, the tiling of incoming data becomes mandatory for 3D CNN acceleration in memory-constrained platforms such as Field Programmable Gate Arrays (FPGA). In this paper, different memory access techniques are explored to reduce the data traffic between on-chip and off-chip memories during the inference stage of a 3D CNN. The most suitable data traffic mode is identified by considering multiple parameters like latency, on-chip memory utilization and off-chip memory access. A parameterized and modular design approach for 3D CNNs has been implemented on an FPGA, where the input and weight data mapping modules are designed to minimize the on-chip memory requirements. These modules are parameterized for variable tiling sizes and different memory access modes while the main computation is performed on a systolic-array-based pipelined architecture. The experiments conducted on three widely adopted 3D networks, I3D, C3D, and R(2+1)D, have shown 16%, 28%, and 10% improvement in latency respectively. The proposed methodology also results in a lower energy dissipation profile.