FPGA Accelerated Automotive ADAS Sensor Fusion

Fusion of multi-modal sensor data is crucial to improve the performance of advanced driver assistance and safety systems. Usually, sensors such as radars, lidars, and cameras work individually to provide the time-varying kinematics (also referred to as tracks) of other vehicles and objects in the field of view of the ego vehicle. Further, these individual tracks are fused in a decentralized manner to achieve the fused tracks. For an automotive vehicle, low-latency and high-speed sensor fusion is a requirement to improve the overall safety. Rather than running the entire fusion algorithm in the central processing unit, some portion of the code or the whole fusion algorithm can be accelerated on a field programmable gate array to improve the overall functionality. With this objective, in this paper we propose a dedicated digital signal processing (DSP) architecture to realize the fusion algorithm which involves computation of fused state and covariance matrix by making use of matrix-to-matrix multiplications and matrix inversion. The matrix inversion is carried out using an efficient LU decomposition method, and matrix multiplication is realized as a vector-to-vector multiplication DSP architecture. Moreover, a folded DSP architecture is proposed for the state and covariance sub-modules to accelerate the overall functionality. Simulations are presented for two-dimensional constant velocity (CV) model with 4-dimensional state space, three-dimensional CV model with 6-dimensional state space, and three-dimensional constant acceleration (CA) model with 9-dimensional state space. Our results indicate that the proposed architecture is well suited for automotive sensor fusion.