Hybrid Dot-Product Design for FP-Enabled FPGAs

FPGAs are becoming interesting solutions for neural network training acceleration. Efficient implementation of dot-products, as part of matrix-matrix multiply engines, plays a key role towards this. The now de facto standard involves the matrix multiplication on bfloat16 inputs, with all reductions be performed in single-precision arithmetic. We present here a generic hybrid dot-product implementation that: (1) has a user-defined accuracy knob and (2) targets a user-defined logic/DSP ratio. Since our architecture is very specialized to a given target device, we discuss the challenges in generating this architecture automatically.