Variance Predictions in VAMP/UAMP with Right Rotationally Invariant Measurement Matrices for niid Generalized Linear Models

In the Generalized Linear Model (GLM), the unknowns and the measurements may be non-identically independent distributed (niid), as, for instance, in the Sparse Bayesian Learning (SBL) problem. The Generalized Approximate Message Passing (GAMP) algorithm performs computationally efficient belief propagation for Bayesian inference. The GAMP algorithm predicts the posterior variances correctly in the case of measurement matrices with (n)iid entries. In order to cover more ill-conditioned measurement matrices, the (right) rotationally invariant model was introduced in which the (right) singular vectors are Haar distributed. The associated extension of (G)AMP is the Vector (G)AMP ((G)VAMP) algorithm, which yields correct posterior variance predictions in the case of iid unknowns and measurements. However, due to averaging operations, these predictions become inexact in the case of niid unknowns and/or measurements. In this paper we apply Haar Large System Analysis (LSA) to characterize the variance prediction errors that can occur. We also introduce Unitary AMP (UAMP), which can continue to yield correct results with AMP style complexity.