Consistent normalized least mean square filtering with noisy data matrix

When the ordinary least squares method is applied to the parameter estimation problem with noisy data matrix, it is well-known that the estimates turn out to be biased. While this bias term can be somewhat reduced by the use of models of higher order, or by requiring a high signal-to-noise ratio (SNR), it can never be completely removed. Consistent estimates can be obtained by means of the instrumental variable method (IVM),or the total/data least squares method (TLS/DLS). In the adaptive setting for the such problem, a variety of least-mean-squares (LMS)-type algorithms have been researched rather than their recursive versions of IVM or TLS/DLS that cost considerable computations. Motivated by these observations, we propose a consistent LMS-type algorithm for the data least square estimation problem. This novel approach is based on the geometry of the mean squared error (MSE) function, rendering the step-size normalization and the heuristic filtered estimation of the noise variance, respectively, for fast convergence and robustness to stochastic noise. Monte Carlo simulations of a zero-forcing adaptive finite-impulse-response (FIR) channel equalizer demonstrate the efficacy of our algorithm.

I. Introduction

The ordinary Least Square Method yields unbiased parameters estimates although the observation vector can be noisy [1], [2]. However, the parameter estimates become biased when the data matrix has noise terms [2], Ch. 7. Equivalently, in the linear regression model, if the regression variables have noise terms, the estimate of the parameter vector loses its consistency. The noise terms in the data matrix may arise, for instance, in system identification or adaptive filtering, where model-order deficiencies or signal noises yield an overdetermined set of equations whose coefficients are noisy [2], Ch. 7–8. Other examples are inverse plant estimation, impulse response estimation, adaptive inverse control, adaptive channel equalization, adaptive infinite-impulse-response (IIR) filtering using the equation-error method, weight-vector updating in neural networks, etc.