Generalized Least Squares for ESPRIT-Type Direction of Arrival Estimation

The key task in ESPRIT-based parameter estimation is finding the solution to the shift invariance equation (SIE), which is often an overdetermined, linear system of equations. Additional structure is imposed if the two selection matrices, applied to an estimate of the signal subspace, overlap such that the subspace estimation errors on both sides of the SIE are highly correlated. In this letter, we propose a novel SIE solution for Standard ESPRIT and Unitary ESPRIT based on generalized least squares (GLS), assuming a uniform linear array (ULA) and maximum subarray overlap. GLS directly incorporates the statistics of the subspace estimation error via its covariance matrix, which is found analytically by a first-order perturbation expansion. As the subspace error covariance matrix is not invertible, we introduce a regularization with a clever choice of the regularization parameter. The resulting GLS-based Standard ESPRIT and Unitary ESPRIT algorithms achieve a superior performance over existing ESPRIT-type methods and almost attain the Cramér-Rao bound (CRB).

I. Introduction

Estimating the directions of arrival (DOAs) of noise-corrupted signals has long been a fundamental problem in signal processing and is required in many applications, such as radar, sonar, and wireless communications. Due to their low complexity and high-resolution capabilities, ESPRIT-type algorithms [1], [2] are among the most popular subspace-based parameter estimation schemes. After the subspace estimation, ESPRIT-based algorithms solve a highly structured linear system of equations, termed shift invariance equation (SIE). The structure is imposed by applying two selection matrices with potential overlap to an estimate of the signal subspace in which case the perturbations on both sides of the SIE are highly correlated. The SIE is usually solved by means of least squares (LS) [1], total least squares (TLS) [3], or structured least squares (SLS) [4].