Exploring the implementation of LSTM inference on FPGA

Long Short-Term Memory (LSTM) is a widely used deep neural network architecture in sequence modeling and temporal series forecasting. Although the main advantages that LSTM provides in terms of predictive capacity, it is well known in the literature that its performance in inference times and resource consumption is very high, so its application to low-resource environments is an issue. In this article, the performance of a High-Level Synthesis (HLS) implementation of the LSTM inference algorithm is analyzed. Two different LSTM implementations, decoupled-LSTM (d-LSTM) and coupled-LSTM (c-LSTM) are compared on the ZCU104 evaluation board, which contains a Xilinx
^®
Zynq
^®
UltraScale+
^™
MPSoC device, and their performance and resource usage are evaluated for different neural network models with varying complexity. The results show that d-LSTM performs better than c-LSTM in terms of inference time, but is more resource-intensive. Furthermore, the FPGA implementation of LSTM outperforms other common devices like GPUs and CPUs. The scalability of d-LSTM concerning sequence length is also explored. A positive linear behavior has been observed in the two lighter models while in the heavier one, the growth is higher than linear. The study highlights the advantages of using FPGA for implementing LSTM inference algorithms and the importance of exploiting parallelism through libraries like the Basic Linear Algebra Subprograms for optimizing performance.