Implementation and Optimization of LeNet-5 Model for Handwritten Digits Recognition on FPGAs using Brevitas and FINN

Handwritten digit recognition using deep learning models is an active area of research with wide-ranging applications. However, the high computational requirements of these models limit their deployment on resource-constrained devices. To address this challenge, we propose an optimized implementation of the LeNet-5 model for handwritten digit recognition on FPGAs using the Brevitas and FINN frameworks. We trained the model on the MNIST dataset and quantized it to reduce its memory footprint and improve inference performance. We then used FINN to generate hardware operators and RTL code, which were synthesized for an EFINIX FPGA. Additionally, we used a RISC-V processor and custom HLS VDMAs to interface with the FPGA and enable image loading and result display through GUI. Our approach achieved a classification accuracy of 96.64% on the test set, while consuming significantly lower power than CPU and GPU implementations. Our methodology demonstrates the potential of leveraging FPGA hardware acceleration to enable efficient deployment of deep learning models on resource-constrained devices.